{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Genetic Feature Selection nodes in TPOT\n",
    "\n",
    "TPOT can use evolutionary algorithms to optimize feature selection simultaneously with pipeline optimization. It includes two node search spaces with different feature selection strategies: FSSNode and GeneticFeatureSelectorNode. \n",
    "\n",
    "1. FSSNode - (Feature Set Selector) This node is useful if you have a list of predefined feature sets you want to select from. Each FeatureSetSelector Node will select a single group of features to be passed to the next step in the pipeline. Note that FSSNode does not create its own subset of features and does not mix/match multiple predefined feature sets.\n",
    "\n",
    "2. GeneticFeatureSelectorNode—Whereas the FSSNode selects from a predefined list of subsets of features, this node uses evolutionary algorithms to optimize a novel subset of features from scratch. This is useful where there is no predefined grouping of features. \n",
    "\n",
    "This tutorial focuses on FSSNode. See Tutorial 5 for more information on GeneticFeatureSelectorNode.\n",
    "\n",
    "It may also be beneficial to pair these search spaces with a secondary objective function to minimize complexity. That would encourage TPOT to try to produce the simplest pipeline with the fewest number of features.\n",
    "\n",
    "tpot.objectives.number_of_nodes_objective - This can be used as an other_objective_function that counts the number of nodes.\n",
    "\n",
    "tpot.objectives.complexity_scorer - This is a scorer that tries to count the total number of learned parameters (number of coefficients, number of nodes in decision trees, etc.).\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Set Selector\n",
    "\n",
    "The FeatureSetSelector is a subclass of sklearn.feature_selection.SelectorMixin that simply returns the manually specified columns. The parameter sel_subset specifies the name or index of the column that it selects. The transform function then simply indexes and returns the selected columns. You can also optionally name the group with the name parameter, though this is only for note keeping and does is not used by the class.\n",
    "\n",
    "\n",
    "    sel_subset: list or int\n",
    "        If X is a dataframe, items in sel_subset list must correspond to column names\n",
    "        If X is a numpy array, items in sel_subset list must correspond to column indexes\n",
    "        int: index of a single column\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original DataFrame\n",
      "   a  b  c  d  e  f\n",
      "0  0  1  2  3  4  5\n",
      "1  0  1  2  3  4  5\n",
      "2  0  1  2  3  4  5\n",
      "3  0  1  2  3  4  5\n",
      "4  0  1  2  3  4  5\n",
      "5  0  1  2  3  4  5\n",
      "6  0  1  2  3  4  5\n",
      "7  0  1  2  3  4  5\n",
      "8  0  1  2  3  4  5\n",
      "9  0  1  2  3  4  5\n",
      "Transformed Data\n",
      "[[0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]\n",
      " [0 1 2]]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/tpotenv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import tpot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "#make a dataframe with columns a,b,c,d,e,f\n",
    "\n",
    "#numpy array where columns are 1,2,3,4,5,6\n",
    "data = np.repeat([np.arange(6)],10,0)\n",
    "\n",
    "df = pd.DataFrame(data,columns=['a','b','c','d','e','f'])\n",
    "fss = tpot.builtin_modules.FeatureSetSelector(name='test',sel_subset=['a','b','c'])\n",
    "\n",
    "print(\"original DataFrame\")\n",
    "print(df)\n",
    "print(\"Transformed Data\")\n",
    "print(fss.fit_transform(df))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FSSNode\n",
    "\n",
    "The `FSSNode` is a node search space that simply selects one feature set from a list of feature sets. This works identically to the EstimatorNode, but provides a easier interface for defining the feature sets.\n",
    "\n",
    "Note that the FSS is only well defined when used as the first step in a pipeline. This is because downstream nodes will receive different transformations of the data such that the original indexes no longer correspond to the same columns in the transformed data.\n",
    "\n",
    "The `FSSNode` takes in a single parameter `subsets` which defines the groups of features. There are four ways of defining the subsets. \n",
    "\n",
    "    subsets : str or list, default=None\n",
    "            Sets the subsets that the FeatureSetSeletor will select from if set as an option in one of the configuration dictionaries. \n",
    "            Features are defined by column names if using a Pandas data frame, or ints corresponding to indexes if using numpy arrays.\n",
    "            - str : If a string, it is assumed to be a path to a csv file with the subsets. \n",
    "                The first column is assumed to be the name of the subset and the remaining columns are the features in the subset.\n",
    "            - list or np.ndarray : If a list or np.ndarray, it is assumed to be a list of subsets (i.e a list of lists).\n",
    "            - dict : A dictionary where keys are the names of the subsets and the values are the list of features.\n",
    "            - int : If an int, it is assumed to be the number of subsets to generate. Each subset will contain one feature.\n",
    "            - None : If None, each column will be treated as a subset. One column will be selected per subset.\n",
    "\n",
    "\n",
    "Lets say you want to have three groups of features, each with three columns each. The following examples are equivalent:\n",
    "\n",
    "### str\n",
    "\n",
    "sel_subsets=simple_fss.csv\n",
    "\n",
    "\n",
    "    \\# simple_fss.csv\n",
    "    group_one, 1,2,3\n",
    "    group_two, 4,5,6\n",
    "    group_three, 7,8,9\n",
    "\n",
    "\n",
    "### dict\n",
    "\n",
    "\n",
    "sel_subsets = { \"group_one\" :  [1,2,3],\n",
    "            \"group_two\" :  [4,5,6],\n",
    "            \"group_three\" :  [7,8,9],\n",
    "            }\n",
    "\n",
    "\n",
    "### list\n",
    "\n",
    "\n",
    "sel_subsets = [[1,2,3],\n",
    "[4,5,6],\n",
    "[7,8,9]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examples\n",
    "\n",
    "For these examples, we create a dummy dataset where the first six columns are informative and the rest are uninformative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "      <th>g</th>\n",
       "      <th>h</th>\n",
       "      <th>i</th>\n",
       "      <th>j</th>\n",
       "      <th>k</th>\n",
       "      <th>l</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.315814</td>\n",
       "      <td>-3.427720</td>\n",
       "      <td>-1.314654</td>\n",
       "      <td>-1.508737</td>\n",
       "      <td>-0.300932</td>\n",
       "      <td>0.089448</td>\n",
       "      <td>0.327651</td>\n",
       "      <td>0.329022</td>\n",
       "      <td>0.857495</td>\n",
       "      <td>0.734238</td>\n",
       "      <td>0.257218</td>\n",
       "      <td>0.652350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.191001</td>\n",
       "      <td>-1.396922</td>\n",
       "      <td>0.149488</td>\n",
       "      <td>-1.730145</td>\n",
       "      <td>-0.394932</td>\n",
       "      <td>0.519712</td>\n",
       "      <td>0.807762</td>\n",
       "      <td>0.509823</td>\n",
       "      <td>0.876159</td>\n",
       "      <td>0.002806</td>\n",
       "      <td>0.449828</td>\n",
       "      <td>0.671350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.661264</td>\n",
       "      <td>-0.981737</td>\n",
       "      <td>0.703879</td>\n",
       "      <td>0.730321</td>\n",
       "      <td>-2.750405</td>\n",
       "      <td>0.396581</td>\n",
       "      <td>0.380302</td>\n",
       "      <td>0.532604</td>\n",
       "      <td>0.877129</td>\n",
       "      <td>0.610919</td>\n",
       "      <td>0.780108</td>\n",
       "      <td>0.625689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.445936</td>\n",
       "      <td>0.354237</td>\n",
       "      <td>0.779040</td>\n",
       "      <td>1.288014</td>\n",
       "      <td>2.397133</td>\n",
       "      <td>0.186324</td>\n",
       "      <td>0.544191</td>\n",
       "      <td>0.465419</td>\n",
       "      <td>0.588535</td>\n",
       "      <td>0.919575</td>\n",
       "      <td>0.513460</td>\n",
       "      <td>0.831546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.989027</td>\n",
       "      <td>-1.824787</td>\n",
       "      <td>-1.448234</td>\n",
       "      <td>1.546442</td>\n",
       "      <td>1.643775</td>\n",
       "      <td>0.167975</td>\n",
       "      <td>0.188238</td>\n",
       "      <td>0.024149</td>\n",
       "      <td>0.544878</td>\n",
       "      <td>0.834503</td>\n",
       "      <td>0.877869</td>\n",
       "      <td>0.278330</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          a         b         c         d         e         f         g  \\\n",
       "0  2.315814 -3.427720 -1.314654 -1.508737 -0.300932  0.089448  0.327651   \n",
       "1 -0.191001 -1.396922  0.149488 -1.730145 -0.394932  0.519712  0.807762   \n",
       "2  0.661264 -0.981737  0.703879  0.730321 -2.750405  0.396581  0.380302   \n",
       "3  1.445936  0.354237  0.779040  1.288014  2.397133  0.186324  0.544191   \n",
       "4 -0.989027 -1.824787 -1.448234  1.546442  1.643775  0.167975  0.188238   \n",
       "\n",
       "          h         i         j         k         l  \n",
       "0  0.329022  0.857495  0.734238  0.257218  0.652350  \n",
       "1  0.509823  0.876159  0.002806  0.449828  0.671350  \n",
       "2  0.532604  0.877129  0.610919  0.780108  0.625689  \n",
       "3  0.465419  0.588535  0.919575  0.513460  0.831546  \n",
       "4  0.024149  0.544878  0.834503  0.877869  0.278330  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tpot\n",
    "import sklearn.datasets\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tpot\n",
    "import sklearn.datasets\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "from tpot.search_spaces.nodes import *\n",
    "from tpot.search_spaces.pipelines import *\n",
    "from tpot.config import get_search_space\n",
    "\n",
    "\n",
    "X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=6, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)\n",
    "X = np.hstack([X, np.random.rand(X.shape[0],6)]) #add six uninformative features\n",
    "X = pd.DataFrame(X, columns=['a','b','c','d','e','f','g','h','i', 'j', 'k', 'l']) # a, b ,c the rest are uninformative\n",
    "X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)\n",
    "\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets say that either based on prior knowledge or interest, we know that the features can be grouped as follows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "subsets = { \"group_one\" :  ['a','b','c',],\n",
    "            \"group_two\" :  ['d','e','f'],\n",
    "            \"group_three\" :  ['g','h','i'],\n",
    "            \"group_four\" :  ['j','k','l'],\n",
    "            }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create an FSSNode that will select from this subset. Each node in a pipeline only selects one subset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "fss_search_space = FSSNode(subsets=subsets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we randomly sample from this search space, we can see that we get a single selector that selects one of the predefined sets. In this case, it selects groups two, which includes ['d', 'e', 'f']. (A random seed was set in the generate function so that the same group would be selected when rerunning the notebook.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;FeatureSetSelector<span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fss_selector = fss_search_space.generate(rng=1).export_pipeline()\n",
    "fss_selector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.315442</td>\n",
       "      <td>-1.039258</td>\n",
       "      <td>0.194516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>-1.908995</td>\n",
       "      <td>-0.953551</td>\n",
       "      <td>-1.430472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>0.181162</td>\n",
       "      <td>1.022858</td>\n",
       "      <td>-2.289700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2.825765</td>\n",
       "      <td>-1.205520</td>\n",
       "      <td>1.147791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>-2.300481</td>\n",
       "      <td>1.023173</td>\n",
       "      <td>0.449162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>-1.793062</td>\n",
       "      <td>2.209649</td>\n",
       "      <td>-0.045031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>-0.221409</td>\n",
       "      <td>1.688750</td>\n",
       "      <td>0.069356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>0.141471</td>\n",
       "      <td>-1.880294</td>\n",
       "      <td>1.984397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>-0.359952</td>\n",
       "      <td>1.141758</td>\n",
       "      <td>2.019301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.171312</td>\n",
       "      <td>0.079332</td>\n",
       "      <td>0.178522</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>750 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            d         e         f\n",
       "162  1.315442 -1.039258  0.194516\n",
       "168 -1.908995 -0.953551 -1.430472\n",
       "214  0.181162  1.022858 -2.289700\n",
       "895  2.825765 -1.205520  1.147791\n",
       "154 -2.300481  1.023173  0.449162\n",
       "..        ...       ...       ...\n",
       "32  -1.793062  2.209649 -0.045031\n",
       "829 -0.221409  1.688750  0.069356\n",
       "176  0.141471 -1.880294  1.984397\n",
       "124 -0.359952  1.141758  2.019301\n",
       "35   0.171312  0.079332  0.178522\n",
       "\n",
       "[750 rows x 3 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fss_selector.set_output(transform=\"pandas\") #by default sklearn selectors return numpy arrays. this will make it return pandas dataframes\n",
    "fss_selector.fit(X_train)\n",
    "fss_selector.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Under the hood, mutation will randomly select another feature set and crossover will swap the feature sets selected by two individuals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-2 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-2 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-2 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-2 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-2 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;FeatureSetSelector<span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "FeatureSetSelector(name='group_two', sel_subset=['d', 'e', 'f'])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ind1 = fss_search_space.generate(rng=1)\n",
    "ind1.export_pipeline()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-3 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-3 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-3 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-3 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-3 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-3 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-3 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-3 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-3 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-3 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-3 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureSetSelector(name=&#x27;group_four&#x27;, sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;, &#x27;l&#x27;])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;FeatureSetSelector<span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_four&#x27;, sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;, &#x27;l&#x27;])</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "FeatureSetSelector(name='group_four', sel_subset=['j', 'k', 'l'])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ind1.mutate()\n",
    "ind1.export_pipeline()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now use this when defining our pipelines. \n",
    "For this first example, we will construct a simple linear pipeline where the first step is a feature set selector, and the second is a classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ketrong/Desktop/tpotvalidation/tpot/tpot/tpot_estimator/estimator.py:456: UserWarning: Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\n",
      "  warnings.warn(\"Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\")\n",
      "Generation: 100%|██████████| 5/5 [00:36<00:00,  7.26s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.926166142557652\n"
     ]
    }
   ],
   "source": [
    "\n",
    "classification_search_space = get_search_space([\"RandomForestClassifier\"])\n",
    "fss_and_classifier_search_space = SequentialPipeline([fss_search_space, classification_search_space])\n",
    "\n",
    "\n",
    "est = tpot.TPOTEstimator(generations=5, \n",
    "                            scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n",
    "                            scorers_weights=[1.0, -1.0],\n",
    "                            n_jobs=32,\n",
    "                            classification=True,\n",
    "                            search_space = fss_and_classifier_search_space,\n",
    "                            verbose=1,\n",
    "                            )\n",
    "\n",
    "\n",
    "scorer = sklearn.metrics.get_scorer('roc_auc_ovr')\n",
    "est.fit(X_train, y_train)\n",
    "print(scorer(est, X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-4 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-4 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-4 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-4 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-4 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-4 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-4 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-4 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-4 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-4 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-4 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-4 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-4 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-4 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-4 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-4 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-4 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-4 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-4\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                 FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                    sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(max_features=0.30141491087,\n",
       "                                        min_samples_leaf=4,\n",
       "                                        min_samples_split=17, n_estimators=128,\n",
       "                                        n_jobs=1))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;Pipeline<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for Pipeline</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                 FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                    sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(max_features=0.30141491087,\n",
       "                                        min_samples_leaf=4,\n",
       "                                        min_samples_split=17, n_estimators=128,\n",
       "                                        n_jobs=1))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">FeatureSetSelector</label><div class=\"sk-toggleable__content fitted\"><pre>FeatureSetSelector(name=&#x27;group_one&#x27;, sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-6\" type=\"checkbox\" ><label for=\"sk-estimator-id-6\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;RandomForestClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.RandomForestClassifier.html\">?<span>Documentation for RandomForestClassifier</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>RandomForestClassifier(max_features=0.30141491087, min_samples_leaf=4,\n",
       "                       min_samples_split=17, n_estimators=128, n_jobs=1)</pre></div> </div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('featuresetselector',\n",
       "                 FeatureSetSelector(name='group_one',\n",
       "                                    sel_subset=['a', 'b', 'c'])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(max_features=0.30141491087,\n",
       "                                        min_samples_leaf=4,\n",
       "                                        min_samples_split=17, n_estimators=128,\n",
       "                                        n_jobs=1))])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "est.fitted_pipeline_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this setup TPOT is able to identify one of the subsets used, but the performance is not optimal. In this case we happen to know that multiple feature sets are required. If we want to include multiple features in our pipelines, we will have to modify our search space. There are three options for this.\n",
    "\n",
    "1. UnionPipeline - This allows you to have a fixed number of feature sets selected. If you use a UnionPipeline with two FSSNodes, you will always select two feature sets that are simply concatenated together.\n",
    "2. DynamicUnionPipeline - This space allows multiple FSSNodes to be selected. Unlike UnionPipeline you don't have to specify the number of selected sets, TPOT will identify the number of sets that are optimal. Additionally, with DynamicUnionPipeline, the same feature set cannot be selected twice. Note that while DynamicUnionPipeline can select multiple feature sets, it never mixes two feature sets together.\n",
    "3. GraphSearchPipeline - When set as the leave_search_space, GraphSearchPipeline can also select multiple FSSNodes which act as an input to the rest of the pipeline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  UnionPipeline + FSSNode example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "union_fss_space = UnionPipeline([fss_search_space, fss_search_space])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-5 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-5 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-5 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-5 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-5 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-5 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-5 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-5 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-5 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-5 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-5 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                                   sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])),\n",
       "                               (&#x27;featuresetselector-2&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_three&#x27;,\n",
       "                                                   sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;,\n",
       "                                                               &#x27;i&#x27;]))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" ><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;&nbsp;FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for FeatureUnion</span></a><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                                   sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])),\n",
       "                               (&#x27;featuresetselector-2&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_three&#x27;,\n",
       "                                                   sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;,\n",
       "                                                               &#x27;i&#x27;]))])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featuresetselector-1</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featuresetselector-2</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_three&#x27;, sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;, &#x27;i&#x27;])</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                FeatureSetSelector(name='group_two',\n",
       "                                                   sel_subset=['d', 'e', 'f'])),\n",
       "                               ('featuresetselector-2',\n",
       "                                FeatureSetSelector(name='group_three',\n",
       "                                                   sel_subset=['g', 'h',\n",
       "                                                               'i']))])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this union search space will always select exactly two fss_search_space\n",
    "selector1 = union_fss_space.generate(rng=1).export_pipeline()\n",
    "selector1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "      <th>g</th>\n",
       "      <th>h</th>\n",
       "      <th>i</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.315442</td>\n",
       "      <td>-1.039258</td>\n",
       "      <td>0.194516</td>\n",
       "      <td>0.751175</td>\n",
       "      <td>0.411340</td>\n",
       "      <td>0.824754</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>-1.908995</td>\n",
       "      <td>-0.953551</td>\n",
       "      <td>-1.430472</td>\n",
       "      <td>0.072697</td>\n",
       "      <td>0.875766</td>\n",
       "      <td>0.953255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>0.181162</td>\n",
       "      <td>1.022858</td>\n",
       "      <td>-2.289700</td>\n",
       "      <td>0.135222</td>\n",
       "      <td>0.395847</td>\n",
       "      <td>0.232638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2.825765</td>\n",
       "      <td>-1.205520</td>\n",
       "      <td>1.147791</td>\n",
       "      <td>0.925905</td>\n",
       "      <td>0.486645</td>\n",
       "      <td>0.710991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>-2.300481</td>\n",
       "      <td>1.023173</td>\n",
       "      <td>0.449162</td>\n",
       "      <td>0.645161</td>\n",
       "      <td>0.131657</td>\n",
       "      <td>0.863514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>-1.793062</td>\n",
       "      <td>2.209649</td>\n",
       "      <td>-0.045031</td>\n",
       "      <td>0.502947</td>\n",
       "      <td>0.994603</td>\n",
       "      <td>0.280062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>-0.221409</td>\n",
       "      <td>1.688750</td>\n",
       "      <td>0.069356</td>\n",
       "      <td>0.328066</td>\n",
       "      <td>0.102381</td>\n",
       "      <td>0.492280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>0.141471</td>\n",
       "      <td>-1.880294</td>\n",
       "      <td>1.984397</td>\n",
       "      <td>0.365550</td>\n",
       "      <td>0.465859</td>\n",
       "      <td>0.974601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>-0.359952</td>\n",
       "      <td>1.141758</td>\n",
       "      <td>2.019301</td>\n",
       "      <td>0.329380</td>\n",
       "      <td>0.718647</td>\n",
       "      <td>0.365507</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.171312</td>\n",
       "      <td>0.079332</td>\n",
       "      <td>0.178522</td>\n",
       "      <td>0.215759</td>\n",
       "      <td>0.546279</td>\n",
       "      <td>0.662928</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>750 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            d         e         f         g         h         i\n",
       "162  1.315442 -1.039258  0.194516  0.751175  0.411340  0.824754\n",
       "168 -1.908995 -0.953551 -1.430472  0.072697  0.875766  0.953255\n",
       "214  0.181162  1.022858 -2.289700  0.135222  0.395847  0.232638\n",
       "895  2.825765 -1.205520  1.147791  0.925905  0.486645  0.710991\n",
       "154 -2.300481  1.023173  0.449162  0.645161  0.131657  0.863514\n",
       "..        ...       ...       ...       ...       ...       ...\n",
       "32  -1.793062  2.209649 -0.045031  0.502947  0.994603  0.280062\n",
       "829 -0.221409  1.688750  0.069356  0.328066  0.102381  0.492280\n",
       "176  0.141471 -1.880294  1.984397  0.365550  0.465859  0.974601\n",
       "124 -0.359952  1.141758  2.019301  0.329380  0.718647  0.365507\n",
       "35   0.171312  0.079332  0.178522  0.215759  0.546279  0.662928\n",
       "\n",
       "[750 rows x 6 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selector1.set_output(transform=\"pandas\") \n",
    "selector1.fit(X_train)\n",
    "selector1.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  DynamicUnionPipeline + FSSNode example\n",
    "The dynamic union pipeline may select a variable number of feature sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-6 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-6 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-6 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-6 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-6 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-6 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-6 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-6 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-6 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-6 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_three&#x27;,\n",
       "                                                   sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;,\n",
       "                                                               &#x27;i&#x27;]))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" ><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;&nbsp;FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for FeatureUnion</span></a><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_three&#x27;,\n",
       "                                                   sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;,\n",
       "                                                               &#x27;i&#x27;]))])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featuresetselector</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-11\" type=\"checkbox\" ><label for=\"sk-estimator-id-11\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_three&#x27;, sel_subset=[&#x27;g&#x27;, &#x27;h&#x27;, &#x27;i&#x27;])</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "FeatureUnion(transformer_list=[('featuresetselector',\n",
       "                                FeatureSetSelector(name='group_three',\n",
       "                                                   sel_subset=['g', 'h',\n",
       "                                                               'i']))])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dynamic_fss_space = DynamicUnionPipeline(fss_search_space)\n",
    "dynamic_fss_space.generate(rng=1).export_pipeline()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-7 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-7 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-7 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-7 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-7 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-7 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-7 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-7 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-7 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-7 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-7 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-7 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-7 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-7 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-7 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-7 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-7 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-7 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-7\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                   sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])),\n",
       "                               (&#x27;featuresetselector-2&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_four&#x27;,\n",
       "                                                   sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;,\n",
       "                                                               &#x27;l&#x27;]))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-12\" type=\"checkbox\" ><label for=\"sk-estimator-id-12\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;&nbsp;FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for FeatureUnion</span></a><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                   sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])),\n",
       "                               (&#x27;featuresetselector-2&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_four&#x27;,\n",
       "                                                   sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;,\n",
       "                                                               &#x27;l&#x27;]))])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featuresetselector-1</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-13\" type=\"checkbox\" ><label for=\"sk-estimator-id-13\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_one&#x27;, sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featuresetselector-2</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-14\" type=\"checkbox\" ><label for=\"sk-estimator-id-14\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_four&#x27;, sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;, &#x27;l&#x27;])</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                FeatureSetSelector(name='group_one',\n",
       "                                                   sel_subset=['a', 'b', 'c'])),\n",
       "                               ('featuresetselector-2',\n",
       "                                FeatureSetSelector(name='group_four',\n",
       "                                                   sel_subset=['j', 'k',\n",
       "                                                               'l']))])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dynamic_fss_space.generate(rng=3).export_pipeline()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### GraphSearchPipeline + FSSNode example\n",
    "\n",
    "FSSNodes must be set as the leaf search space as they act as the inputs to the pipeline.\n",
    "\n",
    "Here is an example pipeline from this search space that utilizes two feature sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnYAAAHWCAYAAAD6oMSKAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXWFJREFUeJzt3QlY1OX2wPEDDCqLsogKLrhi7vuOlt2sW2bezNSsW6bt+75aeTO1xRa7LVqZZrutN/9ZWbZYLrmgae6IKIpCIgiyCQP8n/Pe4DLKGArMb2b4fp6Hx3r5zcwBlN+Z97zveX1KSkpKBAAAAB7P1+oAAAAAUD1I7AAAALwEiR0AAICXILEDAADwEiR2AAAAXoLEDgAAwEuQ2AEAAHgJEjsAAAAvQWIHAADgJUjsAAAAvASJHQAAgJcgsQMAAPASJHYAAABegsQOAADAS5DYAQAAeAkSOwAAAC9BYgcAAOAlSOwAAAC8BIkdAACAlyCxAwAA8BIkdgAAAF6CxA4AAMBLkNgBAAB4CRI7AAAAL0FiBwAA4CVI7AAAALwEiR0AAICXILEDAADwEjarAwCA6lRUVCTp6emSmppqPg6lpMixvDwpLioSXz8/qRsQII0iI6VJkybmIzw8XPz8/KwOGwCqhU9JSUlJ9TwVAFgnIyNDNm7cKL+vXy/5OTlSYrdLcF6ehKSni7/dLr4lJVLs4yOFNptkhodLdkCA+NhsUi8oSLr26iXdu3eXsLAwq78MAKgSEjsAHu3AgQOycvlySYyPF//cXIlO2idR6ekSkpMj/kVFTh9X6OcnmUFBcjA8XJKiW0hhYKC0jomR2CFDJCoqyqVfAwBUFxI7AB7JbrfLihUrZO2KFRKclibt9iZJ87Q08SsuPuXnKvL1lf0REbKrZbRkR0RI39hYiY2NFZuN1SoAPAuJHQCPk5KSIosXLZKM/cnSIT5eYpKTTam1qrRUG9+smWyPiZHw5s1k+MiREhkZWS0xA4ArkNgB8Ch79+6VzxculMADB6X3tm3SIDe32l8jKzBQ4jp2lNymTWXUuLHSsmXLan8NAKgJJHYAPCqp+/SDD6Th3iTpt3Wr2E6j7FpZdl9fWd25k6RHR8vo8eNJ7gB4BPrYAfCY8qvO1IXvTZIBW7bUaFKn9PkHbt4i4UlJ8vnCj8zrA4C7I7ED4BEbJXRNnZZf+2/dWi3r6SpDX6f/lq0ScPCAfLVokYkDANwZiR0At6e7X3WjhK6pq+mZuuPp6/Xeuk3Sk5Nl5cqVLn1tADhVJHYA3L5PnbY00d2vNbFRojJCcnPljJ3xsmb5cjl48KAlMQBAZZDYAXBr2nxY+9RpSxMrtU9ONnGsWL7c0jgA4GRI7AC49TFheqKENh921bo6Z/T12+5NksSdO01cAOCOSOwAuC09+1WPCdMTJdxBi7Q0seXmyqZNm6wOBQAqRGIHwC0VFRXJ7+vXm7NfT+eYsJqgcbTct082xcWZ+ADA3ZDYAXBL6enpkp+TI1Hp6eJOog7/Ny6NDwDcDYkdAKdsNpv06NGj7CMvL++Un+OZZ545rddOTU2VErtdQrOzHcZfTtorw9fHyYj1cXLJbxtkX37+SZ/njf37qvT4fr+ucvj/kJwcE5fGdzKzZs2SgoICqarffvtNBgwYIF26dJFevXrJTz/9VOXnBOC9bFYHAMB9hYaGmsSiKjSxu//++0/pMVrm1MQpOC/PoW/d+qwsWZ2ZKV/06Cn+vr6ScuyYBPid/P3pG/v3y3XNW5z244/nX1Rk4tL4NNk6WWJ37bXXSp06dSr1vMXFxeLre2IsQUFB8t5770nbtm1l69atMmLECNm9e/cpxQyg9mDGDsApWbJkiQwcOFB69uwp//znP8tmpa6//nrp3bu3dO7cWZ599lkzNnnyZDly5IiZ7bvxxhtlz5490qdPn7Lnuvfee+Wtt94y/92qVSt58MEHzfP+8MMP8tknn8iz8+fLRevXy4w/E5lDBQUSZvM3SZmKrFtXQmz+5r9/yciQsRt/k39sWC/37tguBcXF8vyePXLUbpeRG9bLY7viT/nxx3t9/z4zy/fU3Lky/803y8anT58uXbt2lW7duskLL7wgr7zyium/N2jQIBk5cqS55p133jHXaDI4c+ZMM6bfDx277LLLpFOnThXOiMbExJikTnXs2FGys7NZ3wfAKWbsADhVmpQpTcieeuopk5Ro4hUQECCPPfaYvPHGG3LLLbeYz4WHh5tjt4YMGSLjxo0zCc9rr71WNuuniczJtGjRQjZs2CDbtm2TNWvWyPQLLpA+iXvkvh075Mf0dIkNDZWXkvbKBXHrJDY0TP7RuLF0rV9f0gsLZe7+/fJ2l65Sz89PXty7Rz5KSZG7W7WSD1MOyqKevczzZ9vtp/T4fzZtWhbb8owMM8P3afcesr5NG5m6do1s3rxZkpKSzPdj3bp1UrduXbP2Tr8P+n3SkyqCg4MlOTlZ/vWvf8natWslMDDQJHx/+9vfpGHDhuZr1Rk5TQr/yn/+8x+TPPv5+VXxJwvAW5HYAah0KfbLL780rT50xk4dO3ZMLrzwQvPfH3zwgcydO9fMJu3fv1+2b99uErVTMWbMGPPn999/Lwm7d8tDiYkSUFAg+UXF0iU4WM4OD5f/9Owlq48ckZWZR2Ti5s3yYocOUlBSLDtyc2Tspo3m8TrbNjQ8/ITnD7bZTvvxy49kyE/pGbIua4Pkbd0iuX5+snPnTlm+fLlMnDjRJHVKk7rjaUJ3zjnnlH3u0ksvNY/7xz/+Ie3bt69UUqflVy1pf/3116f0PQVQu5DYAag0XQemidz8+fNPSDq0/Lhq1SoJCQkxiYsmfRVtxtDnKHX8NTqbVfo6Zw0ZIuPDw6VnguN6MpuPj8SGhZmPcJu/LE0/LINDw2RoWLg81b79X34Np/v44hKRW6Oj5ZImTWRD27aSP2SwXHLJJSZBq4rSr/lkdBZQk0Cd/WzXrl2VXg+Ad2ONHYBK05m6H3/8Ufbu3Wv+PysrSxITE+Xo0aOm5NigQQMzW7d06dKyx2jZsHRNWOPGjc3aM71e14p99913Fb6Ozm6tjYuTdLvd/P/hggL5o6BAdufmStKf69BKSkpkZ26ONK1bV3o2qC+rM49I8p87XLXkWrrb1c/HR4r+PLXidB5fanBYqHycmiJ5RUVSYLNJ5tGjkpmZKcOGDTOJbmmSWtoGpX79+ubrVP369TOzkHpihV732WefmXJ1ZegaxlGjRsk999xjyrcAcDLM2AGotEaNGpk1daNHjzYJh+7i1N2fQ4cONQv7O3ToYDZBDB48uOwxEyZMMBsEzjzzTJkzZ44pJ+oGiejoaDNeEd2AcdWECTJt7lyZlZNjNjs8HdNejpUUy9SEBMn+M1HsHBQsV0Y1NeviprWLkdu2b5PC4mLx8fGRya3bSIt69WRU4yamtUnfkBAZGxl5yo8vdWZYuOzKzTUbLLLid0r4r6tk7PjxMnz4cImLizOtSPz9/U1Z9o477pDrrrtOzj77bFNqXbRokUyZMsV8DzSh1O+JXv9Xaw7VRx99JL/++qtJIvV7rTRJ1PV5AHA8nxL9LQMAbkY3Jnz18ccyYtnPpsWIuyj085MvzzpTho8Zc9J2JwBgBUqxANxSkyZNxEdLnkFB4k40Ho1L4wMAd0MpFoBb0h2k9YKC5GB4uERkZYm7ONjwv3FVtPu1Kg4fPmzWFpanO21Xr15dra8DwLuR2AFwS7rpomuvXvLb4cPSKSlJ/CpoGOxqRb6+srdFC+lVA73kdM1cVU/5AABKsQDcVvfu3aUwMFD2R0TUyPNnHc2SAwcPyh+H/pDCP3fgnsy+iAixBwZWqu8cAFiBxA6A2woLC5PWMTGyq2W0FPv4VOtzayKnLVdESsxpGdqmpPgke8n09RNaRkvr9u1NXADgjkjsALi12CFDJDsiQuKbNavR1ykqspu+fM7sbNbMxBFbrpULALgbEjsAbi0qKkr6xsbK9pgYyarEKQ2V5W+zSZ06/z0GrFRubo7kV3BiRmZgoOxoHyP9Bg828QCAuyKxA+D2YmNjJax5M4nr2FHsvr7Vehauj4/j82UeOeJQktXXi+vUUcKbNZNBgwZV22sDQE0gsQPg9vSM2QtHjpTcpk1ldedO1bbezubnZ45BK6+ouMic8qD0dfT18qKayvCRI00cAODOSOwAeITIyEgZNW6spEdHy6ounatt5i4oMFDq1v3f0WEqLy9XcgoKzOvo6+nr6usDgLvjSDEAHmXv3r3y+cKPJPDAAem9bZs0yM2t8nMWFRXJH4cOSUnJf3vl5TRoIDv69JWSNq1l9Pjx0rJly2qIHABqHokdAI+TkpIiixctkoz9ydIhPl5ikpPFt4q/ynLz8iQ984gcaN9e4jt0kOT0dMkrLJR33nlHfKq51QoA1BQSOwAeSXvPrVixQtauWCHBaWnSdm+StEhLO60TKvRECW0+vKVJY0kNCJAVa9fKypUrzUzeBx98IJdddlmNfA0AUN1I7AB4tAMHDsjKFSskcedOseXmSst9+yTqcLqE5OSIf1GR08cV+vlJpp5F2zDcHBOmJ0pEtWghT0yfLjt37iy7Ts+E3bx5M21OAHgEEjsAXiEjI0M2bdokm+LiJD8nR0rsdgnOy5MG6RlSx24X35JiKfbxlQKbTbLCwyQ7IEB8bDapFxQk3Xr3NseE6YkSH330kYwbN87huUeMGCGLFi2iJAvA7ZHYAfAqWj7V48FSU1PNx6GUFCnIz5ciu138tClxvXrSKDJSmjRpYj50Rs7Pz8/hOTSx0wSvvPnz58vVV1/t4q8GAE4NiR0AHCctLU26dOliEsNS2u9OS7ItWrSwNDYAOBn62AHAcSIiIuT11193GNNzZCdNmiS8FwbgzkjsAKACI0eOlAkTJjiMLV26VObMmWNZTADwVyjFAoATR44cMSXZ5OTksrGgoCDZuHGjtG3b1tLYAKAizNgBgBOhoaEyb948h7GcnByZOHGiFJ9GvzwAqGkkdgBwEuedd57ccMMNDmO//PKLvPjii5bFBADOUIoFgL9w9OhR6d69uyQmJpaN1a1bV3777Tfp0KGDpbEBQHnM2AHAX6hfv77pY1fesWPHzOYKPdoMANwFiR0AVMJZZ50ld955p8PYmjVr5JlnnrEsJgA4HqVYAKikvLw86dGjh8NZsv7+/rJu3TpzJBkAWI0ZOwCopICAAFmwYIH4+v7vV2dhYaFcddVVUlBQYGlsAKBI7ADgFAwYMEDuv/9+hzHtazdt2jTLYgKAUpRiAeAU6caJPn36mLNjS/n5+cmqVaukb9++lsYGoHYjsQOA07Bhwwbp16+fw67Yjh07yvr166VevXqWxgag9qIUCwCnoWfPnvLoo486jG3btu2EMQBwJWbsAOA06caJgQMHSlxcXNmYj4+POZkiNjbW0tgA1E4kdgBQBVu2bJFevXo57Ipt27at2VARFBRkaWwAah9KsQBQBZ07d5YnnnjCYSwhIUEefPBBy2ICUHsxYwcAVVRUVCRDhgwxu2LLW7p0qZxzzjmWxQWg9iGxA4BqEB8fL927dzenU5SKjo6W33//XRo0aGBpbABqD0qxAFANYmJi5Omnn3YYS0pKkrvvvtuymADUPszYAUA1KS4ulmHDhsmPP/7oML548WIZPny4ZXEBqD1I7ACgGu3Zs0e6du0q2dnZZWNRUVHmlIrw8HBLYwPg/SjFAkA1atWqlTz//PMOYwcPHpTbb7/dspgA1B7M2AFANdNfq1p6/eabbxzGP/30U7nkkkssiwuA9yOxA4AakJycLF26dJEjR46UjTVq1MiUZBs3bmxpbAC8F6VYAKgBzZo1k5deeslh7NChQ3LTTTeZGT0AqAkkdgBQQ6644gq5+OKLHcY+++wz+eCDDyyLCYB3oxQLADUoNTXVlGTT0tLKxkJDQ80Zs02bNrU0NgDehxk7AKhBTZo0kdmzZzuM6bq76667jpIsgGpHYgcANezSSy+V8ePHO4x99dVXMm/ePMtiAuCdKMUCgAukp6dL586dJSUlpWysfv365izZli1bWhobAO/BjB0AuICeOvHGG284jB09elQmTZpkjiIDgOpAYgcALjJixAiZOHGiw9gPP/xwwho8ADhdlGIBwIUyMzPNWbL79u0rGwsMDJSNGzdKu3btLI0NgOdjxg4AXCgkJOSETRO5ubly9dVXS1FRkWVxAfAOJHYA4GLDhg2Tm2++2WFsxYoV8sILL1gWEwDvQCkWACyQnZ0tPXr0kISEhLKxunXryvr166VTp06WxgbAczFjBwAWCA4Olrfeekt8fHzKxo4dOyYTJkwQu91uaWwAPBeJHQBYZPDgwXL33Xc7jK1bt06eeuopy2IC4NkoxQKAhfLy8qRXr16yffv2sjGbzSZr1641pVoAOBXM2AGAhQICAuTtt98WPz+/sjEtxWpJVkuzAHAqSOwAwGJ9+/aVBx980GFs06ZNMnXqVMtiAuCZKMUCgBsoKCgwCZ4mdKV8fX1l5cqV0r9/f0tjA+A5SOwAwE3o6ROa3BUWFpaNnXHGGbJhwwZTsgWAv0IpFgDcRPfu3WXKlCkOYzt27JBHHnnEspgAeBZm7ADAjejGiUGDBpldsaW0192yZctkyJAhlsYGwP2R2AGAm9m2bZv07NnTYVds69atzfo7bWwMAM5QigUAN9OxY0eZPn26w1hiYqLcf//9lsUEwDMwYwcAbqioqEiGDh0qy5cvdxj/9ttv5dxzz7UsLgDujcQOANxUQkKCdOvWTXJzc8vGmjdvLps3b5aQkBBLYwPgnijFAoCbatu2rcycOdNhbP/+/XLXXXdZFhMA98aMHQC4seLiYjnvvPPk+++/dxhftGiRXHTRRZbFBcA9kdgBgJtLSkqSLl26yNGjR8vGmjRpIlu2bJGGDRtaGhsA90IpFgDcXHR0tMyaNcthLDU1VW699VbLYgLgnpixAwAPoL+qtfS6ePFih/GPPvpIxowZY1lcANwLiR0AeIiDBw9K586dJSMjo2xMS7FaktXSLABQigUADxEVFSWvvPKKw9jhw4flhhtuMDN6AEBiBwAe5LLLLpPRo0c7jH3xxRfy7rvvWhYTAPdBKRYAPMyhQ4dMSVb/LKUNi7VxsTYwBlB7MWMHAB6mUaNG8tprrzmMZWZmyrXXXktJFqjlSOwAwAONGjVK/vnPfzqMLVmyRN544w3LYgJgPUqxAOChdHesNi4+cOBA2VhwcLBs2rRJWrdubWlsAKzBjB0AeKiwsDCZO3euw1h2drZMnDjRHEUGoPYhsQMAD3bBBReYtXXlLVu2TF5++WXLYgJgHUqxAODhsrKypFu3brJ3796ysYCAAPntt9+kffv2lsYGwLWYsQMAD9egQQOZP3++w1heXp5MmDBBioqKLIsLgOuR2AGAFzj77LPltttucxj79ddf5dlnn7UsJgCuRykWALxETk6O9OjRQ3bt2lU2VqdOHYmLizO7ZwF4P2bsAMBLBAUFyYIFC8TX93+/2gsKCuSqq66SwsJCS2MD4BokdgDgRQYNGiT33HOPw9iGDRtkxowZlsUEwHUoxQKAl8nPz5fevXvL1q1by8ZsNpusXr1aevXqZWlsAGoWM3YA4GXq1asnb7/9tvj5+ZWN2e12U5I9duyYpbEBqFkkdgDghXTGbvLkyQ5jW7ZskSlTplgWE4CaRykWALyUbpwYMGCAWWNXSjdWLF++XAYOHGhpbABqBokdAHix33//3czeld8VGxMTY06lCAwMtDQ2ANWPUiwAeLGuXbvK1KlTHcbi4+PloYcesiwmADWHGTsA8HK6cWLw4MFmV2x5P/zwgzmxAoD3ILEDgFpgx44d5lQKbYVSqlWrVrJp0yapX7++pbEBqD6UYgGgFjjjjDPkySefdBjbs2eP3HvvvZbFBKD6MWMHALVEcXGx/O1vf5Nly5Y5jH/99ddy/vnnWxYXgOpDYgcAtcju3bulW7dukpOTUzbWrFkzs3s2LCzM0tgAVB2lWACoRdq0aSPPPfecw1hycrLccccdlsUEoPowYwcAtYz+2tfS67fffusw/vnnn8vFF19sWVwAqo7EDgBqoX379pked5mZmWVjjRs3NseORUREWBobgNNHKRYAaqEWLVrIiy++6DD2xx9/yE033WRm9AB4JmbsAKCW0l//WnpdtGiRw/gHH3wgl112mWVxATh9JHYAUIulpKRI586dJT09vWwsPDxcNm/eLFFRUZbGBuDUUYoFgFosMjJSZs+e7TCmSd71119PSRbwQCR2AFDLjR071nyU9+WXX8qCBQssiwnA6aEUCwCQtLQ06dKli6SmppaNNWjQwJRkdaMFAM/AjB0AwLQ4ef311x3GsrKyZNKkSZRkAQ9CYgcAMEaOHCkTJkxwGFu6dKnMmTPHspgAnBpKsQCAMkeOHDElWT1mrFRQUJBs3LhR2rZta2lsAP4aM3YAgDKhoaEyb948h7GcnByZOHGiFBcXWxYXgMohsQMAODjvvPPkhhtucBj75ZdfTjipAoD7oRQLADjB0aNHpXv37pKYmFg2VrduXfntt9+kQ4cOlsYGwDlm7AAAJ6hfv77Mnz/fYezYsWNmc4XdbrcsLgAnR2IHAKjQWWedJXfeeafD2Jo1a+SZZ56xLCYAJ0cpFgDgVF5envTo0UN27txZNubv7y9r1641pVoA7oUZOwCAUwEBAeZoMV/f/90uCgsLTUm2oKDA0tgAnIjEDgBwUgMGDJD777/fYUz72j3xxBOWxQSgYpRiAQB/STdO9OnTx5wdW8rPz09WrVolffv2tTQ2AP9DYgcAqJQNGzZIv379HHbFduzYUdavXy/16tWzNDYA/0UpFgBQKT179pRHH33UYWzbtm0njAGwDjN2AIBK040TAwcOlLi4uLIxHx8f+fnnn2Xw4MGWxgaAxA4AcIq2bNkivXr1ctgV27ZtW7OhIigoyNLYgNqOUiwA4JR07txZpk2b5jCWkJAgDzzwgGUxAfgvZuwAAKesqKhIzjzzTFm5cqXD+NKlS+Wcc86xLC6gtiOxAwCclvj4eHP6hJ5OUSo6Olp+//13adCggaWxAbUVpVgAwGmJiYmRp59+2mEsKSlJ7r77bstiAmo7ZuwAAKetuLhYhg0bJj/++KPD+JdffikXXnihZXEBtRWJHQCgSvbs2SNdu3aV7OzssrGoqChzSkV4eLilsQG1DaVYAECVtGrVSl544QWHsYMHD8ptt91mWUxAbcWMHQCgyvRWoqXXr7/+2mH8k08+kdGjR1sWF1DbkNgBAKpFcnKydOnSRY4cOVI2FhERYRoaN27c2NLYgNqCUiwAoFo0a9ZMXnrpJYextLQ0uemmm8yMHoCaR2IHAKg2V1xxhYwaNcph7LPPPpP333/fspiA2oRSLACgWv3xxx/m2DGdrSsVGhpqSrJNmza1NDbA2zFjBwCoVrqebvbs2Q5juu7u2muvpSQL1DASOwBAtbv00ktl/PjxDmO6Y3bevHmWxQTUBpRiAQA1Ij093ZRkU1JSysaCg4PNWbLa+w5A9WPGDgBQI/TUiblz5zqM6ekUkyZNMkeRAah+JHYAgBqjTYs1kStPz5V99dVXLYsJ8GaUYgEANSozM9OcJbtv376ysYCAANm4caPExMRYGhvgbZixAwDUqJCQkBM2TeTl5cnVV18tRUVFlsUFeCMSOwBAjRs2bJjcfPPNDmMrV66U559/3rKYAG9EKRYA4BK6caJHjx6SkJBQNlanTh1Zv3692T0LoOqYsQMAuIS2OnnrrbfEx8enbKygoEAmTJgghYWFlsYGeAsSOwCAywwePFjuvvtuh7G4uDh56qmnLIsJ8CaUYgEALqUbJ3r16iXbt28vG7PZbLJmzRrp2bOnpbEBno4ZOwCAS2mrkwULFoifn1/ZmN1uNyXZY8eOWRob4OlI7AAALtevXz958MEHHcb0qLHHH3/cspgAb0ApFgBgCd040bdvX9m0aVPZmK+vr2mD0r9/f0tjAzwViR0AwDJ6+oQmd+V3xZ5xxhmyYcMGU7IFcGooxQIALNO9e3eZMmWKw9iOHTtk8uTJlsUEeDJm7AAAltKNE4MGDZK1a9eWjWmvu59++knOPPNMS2MDPA2JHQDActu2bTOtTsrvim3durVZf6eNjQFUDqVYAIDlOnbsKNOnT3cYS0xMlPvuu8+ymABPxIwdAMAtFBUVydChQ2X58uUO40uWLJHzzjvPsrgAT0JiBwBwGwkJCdKtWzfJzc0tG2vevLnpcRcaGmppbIAnoBQLAHAbbdu2lZkzZzqM7d+/X+666y7LYgI8CTN2AAC3UlxcLH//+99l6dKlDuNffPGFjBw50pRsyx9HBuB/mLEDALgVPX3izTfflAYNGjiMX3fddXLzzTebkmx0dPQJa/EAMGMHAHBT8+fPl0mTJjn9fI8ePcwJFQD+h8QOAOCW9PZ04YUXytdff13h57WJcV5entStW9eUZ9PT0yU1NdV8HEpJkWN5eVJcVCS+fn5SNyBAGkVGSpMmTcxHeHg45Vx4JZvVAQAAUJGDBw/K7t27T5r4bdmyRbKysuT39eslPydHSux2Cc7Lk5D0dAmw28W3pESKfXyk0GaTHeHhEhcQID42m9QLCpKuvXqZI83CwsJc+nUBNYkZOwCAW7rssstk4cKFFX4uMjJSBg8aJD27dpXAwkKJTtonUenpEpKTI/5FRU6fs9DPTzKDguRgeLgkRbeQwsBAaR0TI7FDhkhUVFQNfjWAazBjBwBwS2lpaSeMaflUz5WN7dtXIrKzpcO6OGl79Kj4FRdX6jk16YvIyjIfnZKSZH9EhOw6fFje27VL+sbGSmxsrNhs3BrhuZixAwC4JW13ctFFF0l+fr75/8aNG8vICy+UZmFhErN9uzTduVOCAwIlNCSkSq+jpdr4Zs1ke0yMhDdvJsNHjjQzgoAnIrEDALitXbt2yQMPPCDr1q2TsRdfLFG5udIxLk4Cs7LM5/38bNKkceNqea2swECJ69hRcps2lVHjxkrLli2r5XkBVyKxAwC4tb1798oHb78tIQkJcsaqVeJXbg1ddSZ2yu7rK6s7d5L06GgZPX48yR08Dg2KAQBuKyUlRT5fuFAiDxyUs3cnSnhwfZ2TKPt8cHBwtb6erbhYBm7eIuFJSfL5wo/M6wOehMQOAOCW7Ha7LF60SAIPHJT+W7eKX0mJBAUGSlRkpISFhUvjxk3M/1c3bZHSf8tWCTh4QL5atMjEAXgKEjsAgFtasWKFZOxPlt7btpmZtPKNiQPq1RNbDTYY1tfrvXWbpCcny8qVK2vsdYDqRmIHAHA7Bw4ckLUrVkiH+HhpkJtrSQwhublyxs54WbN8uWmWDHgCEjsAgNtZuXy5BKelSUxysqVxtE9ONnGsWL7c0jiAyiKxAwC4lYyMDEmMj5d2e5PMejcr6eu33ZskiTt3mrgAd0diBwBwKxs3bhT/3FxpXsHJE1ZokZYmttxc2bRpk9WhAH+JxA4A4DaKiork9/XrzdmvlT0mrKZpHC337ZNNcXEmPsCdcSAeAMBtpKenS35OjkSlpzuMd1z+i8QEBUmRlkYDAuXp9u0lwM9PUo4dkyd2J8j2nBwJsdmked168ljbthJRp4553IM7d8rO3Bz5rEfPk77uq0lJsjA1RfKKimTNgIEnfD7qcLok5OSY+Bo1alTNXzVQfZixAwC4jdTUVCmx2yU0O9thvL7NJot69pLFvXqLv6+PfJByUPTgpJu2bpWhYeHyfZ++Jnm7smlTSS8sNI8pKC6W1ZlHzJ9J+Xknfd3BYWHycfceTj8fkpNj4tL4AHdGYgcAcBuaOAXn5Tn0rTtenwYhkpSXLyszj0ign6+MiYws+1zfkBBpHxRk/nt5Roa59sJGjeSrQydfr9etfn1p/OcsX0X8i4pMXCR2cHckdgAAt3EoJUVCjivDlmcvKZGfM9KlfVCgJOTmSueTHCn2VdohuSAiQi6MaGT+u6oapGeY+AB3RmIHAHAbx/LyxL+CI7yO2u0ycsN6ueS3DdK0bj25tEnkyZ+nuFjWZGaaEmt0QIDYfHxkdxUbHdex26UgP79KzwHUNDZPAADcRnFRUYW960rX2JWnmyi+TTtc4fP8lJ4uWXa7/D1unfn/7KIiM2t3a3TL047Nt6RYijg3Fm6OGTsAgNvw9fOTYh+fSl07KDRUsovs8lm5dW/rMjNlZ06OSeKePaOD/Ni3n/n4tEcP+aqKffGKfXzFz8Z8CNwbiR0AwG3UDQiQwkomTz4+PvJqx07y3eHDcs66tTJ8fZy8c/CABPn5ya9Hjsjg0NCya6PrBYif+JikryKz9u6RIWtWm1k+/XNe8v4Trimw2aROvXpV+OqAmudTovvFAQBwA99//73sWLJEzl316wmfK7TbJTcnR4qKiyUoKEjqnmQXa034buAAOePvf5dzzjnHpa8LnArmlAEAbqNJkyYSp7N2fn6mxUhpQnf06FHJL9eL7lh+vjRu0kT8fF1TeNJ4sgMCTHyAOyOxAwC4DU2cfGw2yQwKkgaHD0t2tiZ0J+5ELZESKSqyi5/vqc3a/Sthl6zPynIYu69VaxkSFnbSx2k8GheJHdwdiR0AwG2Eh4eL+PlJQkCAtDhJ7zl//zrib/M/5ef/V9t2pxXXwYbhUi8o6L/xAW6MzRMAALewcuVKufDCC+XLJUtkT7OmUlRBmdXHx1fqB9eXhg0bms0TrqBx7G3RQrr17i1+fn4ueU3gdJHYAQAstWzZMhk2bJjExsbKkiVLZOPGjZLr7y+Hmzcvu8ZXE7r6DUwptH79+uLroqRO7YuIEHtgoHTr1s1lrwmcLkqxAACX04YMP/zwg0ydOlV+/vlnh89lZmZKfGKiNIyJkcbJydIgMEgCg4JcmsyV0p56CS2jpXX79hL2F+vwAHfAjB0AwKUJnc7KDR482MzSHZ/Uldq6fbvkNW4sWb16SXBwsCVJndrZrJlkR0RI7ODBlrw+cKpI7AAALknoFi9eLAMGDJDzzz/frKerSNOmTWXWrFmydu1aGTJsmOyIaS9ZgYE1FldRcZHk5edJcQUtXTMDA2VH+xjpN3iwREVF1VgMQHWiFAsAqNGEbtGiRabkun79eqfXNW/eXB566CGZNGmS1PvzdAddc7drxw6Jy+ooQzZsEFtxcbXGlpefLxkZGaZ5ioiPNGjQwDQ+1rlBu6+vxHXqKOHNmsmgQYOq9XWBmsSMHQCg2hUXF8snn3wiPXv2lIsvvthpUteyZUt57bXXZNeuXXLzzTeXJXXKZrPJhSNHSm7TprK6c6dKnyFbWdr0+L9JnSqRrKxMOXTokOQVFJjXy4tqKsNHjjRxAJ6CxA4AUG2Kiorkww8/NDtIx4wZY3a4VqRNmzby5ptvSnx8vFx//fVSt27dCq+LjIyUUePGSnp0tKzq0tnMpFWXitqlHCspll/OaC8JDRpIz/79zOsDnoSzYgEAVWa3201CN23aNNmxY4fT62JiYuSRRx6Ryy+//JRmwvbu3SufL/xIAg8ckN7btkmD3Nwqx5x19Kg52aJUToMGsr13HzkQGCAfff65HDx4UD766CMZNWpUlV8LcBUSOwDAaSssLJT33ntPpk+fbsqpznTs2NEkdOPGjTvtJr8pKSmyeNEiydifLB3i4yUmOVl8q3ALy83LkyNHMkyJ90D79hLfoYMkp6fLoq++kj/++MNc07t3b1m3bt1pvwbgaiwcAACcsoKCAnn77bdlxowZkpiY6PS6Ll26yKOPPiqjR4+u8qkNWhadMGmSrFixQtbWqyv7oyKl7d4kaZGWJn6nsbHCp46/pLZsKfvatZO04GBZsXat2a2r5eRSERERVYoZcDVm7AAAlXbs2DGZP3++PPnkk5KUlOT0uh49eshjjz0m//jHP8S3GtfFlTpw4ICsXLFCEnfuFFturrTct0+iDqdLSE6O+JdLzI5X6OcnmUFB5uzXPc2bS1pBgexMTJQVK1eaGcHyoqOjTRPltm3bVnv8QE0hsQMA/KW8vDyZO3euPP3005KcnOz0uj59+piEbsSIES45y1XblWzatEk2xcVJfk6OlNjtEpyXJw3SM6SO3S6+JcVS7OMrBTabZIWHSXZAgPjYbFIvKEi69uol48ePPyGhK6VtTn755ZcaSUyBmkJiBwBwKjc317QjeeaZZ5wmQKp///4yZcoU03zYFQnd8bR8mp6eLqmpqebjUEqKFOTnS5HdLn42m9SpV08aRUaas2b1Izw83JSGzz77bPnpp5+cPu/zzz8vd911l0u/FqAqSOwAACfIzs6W2bNny7PPPlu2kaAiejSYztDp8WBWJHRVpf31dEOHlnZ19m7p0qVmB24pbcOyYcMGs/kD8AQkdgCAMllZWfLKK6/Ic889J4cPH3Z63dChQ01Cp396YkJX0YyfzuAtW7bMfE3l9e3b12yqoFExPAELBwAAcuTIEXPsV6tWreThhx92mtSde+658vPPP8uPP/5oypjekNSp0h27Z511ltx5550On9Nza7UUDXgCZuwAoBbTdWmzZs2SF1980czWOXPBBReYtiUDBw6U2rBRRHf17ty5s2zM39/fJHjdu3e3NDbgr5DYAUAtlJaWZjYGvPTSS2Y9nTMXXXSRSei0HFmb/PrrrxIbG2vOvC2lSd2aNWukTp06lsYGnAylWACoRXTH6P33329KrtqLzllSp8do6caCRYsW1bqkTg0YMMB8n8rTc2+feOIJy2ICKoMZOwCoBfTc05kzZ8qcOXNMqbEiul5uzJgx5uivrl27Sm2nzZi1L9/mzZsd1uKtWrWqVia78AwkdgDgxfbv32+aCr/xxhsmUamINuC97LLLZPLkydKpUyeXx+jOtNVJv379xG63l41p6xOdzaxXr56lsQEVoRQLAF5Ie7HddNNN5jisl19+ucKkTmefrrrqKtm6dau89957JHUV6Nmzp1ljWN62bdtOGAPcBTN2AOBFdu/ebdbOvfXWWw6zTOVpPzZN6B566CFp166dy2P0NIWFhWY3cFxcnEPZWtu+aINmwJ2Q2AGAF4iPj5cZM2bIO++8Y5rtVkRbdkycOFEefPBBad26tctj9GRbtmyRXr16SUFBQdmYzobqhoqgoCBLYwPKoxQLAB5s+/btcuWVV0qHDh3MLF1FSZ2257jllltk165d5txXkrpT17lz5xN2xCYkJMgDDzxgWUxARZixAwAPnUGaNm2aLFy4UJz9GtfF/TfccIPcd9990qxZM5fH6G00aR4yZIjZFVueni97zjnnWBYXUB6JHQB4kNJeap9++qnTawIDA83GiXvvvVciIyNdGl9tKHlro+LyLWOio6Pl999/lwYNGlgaG6AoxQKAB9CF+xdffLE56spZUqdrvbQ0mJiYKM8++yxJXQ2IiYkx7WPKS0pKkrvvvtuymIDymLEDADe2evVqM0O3ePFip9fUr19fbr/9dnN4fUREhEvjq430mLFhw4bJjz/+6DD+5ZdfyoUXXmhZXIAisQMAN7RixQqT0C1ZssTpNSEhISaZu+OOOyQsLMyl8dV2e/bsMadzlD+SLSoqypxSER4ebmlsqN0oxQKAG1m2bJlZiK/90ZwldZo46MYJbUL8r3/9i6TOAnrW7vPPP3/CsW233XabZTEBihk7ALCY/hr+4YcfZOrUqabprTNaZtUNETfffLMpv8L6n9vw4cPlm2++cRj/5JNPZPTo0ZbFhdqNxA4ALKK/fr/99luT0K1cudLpdU2aNDEtS2688Uaa4bqZ5ORk6dKlixw5csQhAdd2NI0bN7Y0NtROlGIBwIKETjdDDBgwQM4//3ynSZ2u2Zo1a5Y5Juyee+4hqXND2h/wpZdechhLS0sz7WaYN4EVmLEDABfRX7eLFi0yM3Tr1693el3z5s3NsV/XXHONaTIM9/+5XnLJJfKf//zHYfzdd9+VK664wrK4UDuR2AGAC9pjfPbZZ2bDgzYYdqZly5by8MMPy4QJE6Ru3boujRFVk5qaakqyOltXKjQ01JRkmzZtamlsqF0oxQJADR5B9eGHH0q3bt1kzJgxTpO6Nm3ayJtvvmlONbj++utJ6jyQroOcPXu2w5iuu7v22mspycKlSOwAoJrZ7XZThtOD48ePH29mbZydYrBgwQLZsWOHTJo0Sfz9/V0eK6rPpZdean7e5X399dcyb948y2JC7UMpFgCqSWFhobz33nsyffp02bVrl9PrOnbsKI888oiMGzdO/Pz8XBojalZ6erpJ6FNSUsrGgoODzVmy2vsOqGnM2AFAFRUUFMjcuXPljDPOkIkTJzpN6nQN1sKFC81N/vLLLyep80LaPPqNN95wGNPTKXRGVtdaAjWNxA4ATtOxY8dkzpw5pqR63XXXSWJiYoXX9ejRw2ye0DV2Y8eOJaHzciNGjDCJXHl6ruyrr75qWUyoPSjFAsApysvLMzN0Tz/9tGlQ60yfPn3kscceMzd6Hx8fl8YIa2VmZpqzZPft21c2FhAQYJJ7fSMA1BQSOwCopNzcXHnttdfkmWeecVhDdbz+/fvLlClTTPNhErraa+nSpXLuuec6jA0aNMgcG8esLWoKpVgA+Au6RmrmzJnSunVrufvuu50mdYMHDzZHhK1atUouuOACkrpabtiwYeZc3/L0lJHnn3/espjg/ZixAwAnsrKy5JVXXpHnnntODh8+7PS6oUOHmpKr/kkyh+PfFOgay4SEhLKxOnXqmJNHdPcsUN1I7ADgONpY9t///rc5pzUjI8PpdVpme/TRR2XIkCEujQ+eZfny5XLmmWc6NCru3bu3mdmldyGqG6VYACjXg0xn3vRoL10j5yyp0zKrltS07EpSh7+iJXot4ZcXFxcnTz31lGUxwXsxYweg1tPzPXXd00svvWRKZ85cdNFFZoaub9++Lo0P3rGTulevXrJ9+/ayMZvNJmvWrJGePXtaGhu8C4kdgFp9cLuun9P+Yjk5OU6vGzVqlEnouAGjKjSJ012xeoZwKW2JsnbtWs4HRrWhFAug1jl48KApjekuV93tWlFSp5sgxowZY/qOaXNhkjpUVb9+/eTBBx90GNNTSB5//HHLYoL3YcYOQK2xf/9+01RYj3zSUyMq4uvrK5dddplMnjxZOnXq5PIY4f3Hz2mCp28Yyv+d0zWb2v8QqCoSOwBeb+/evWah+rx588yNtSLaMPaKK66Qhx9+2Jz5CtQUTep0nWZhYWHZmP6d27BhgzmdAqgKSrEAvNbu3bvNGa7t2rUzZ7pWlNTpAnY911MXtS9YsICkDjWue/fuZtd1eTt27DCzxEBVMWMHwOvEx8fLjBkz5J133nFYqF6e9g+bOHGiWfOka+0AV7Lb7WYjhW6cKL+u86effjI974DTRWIHwGvorNv06dPl/fffl+Li4gqv0a7/1157rTzwwAMSHR3t8hiBUtu2bTObcsqv99Q3GZs2bZLg4GBLY4PnohQLwONt3rzZbHjQzQ7vvvtuhUldvXr15PbbbzflWT0mjKQOVuvYsaN5I1JeYmKi3HfffZbFBM/HjB0Aj16E/sQTT8inn37q9BpdjK4Hsd97770SGRnp0viAv6JLBfSMYT12rLwlS5bIeeedZ1lc8FwkdgA8jh7HpAndF1984fSaoKAgufXWW02/usaNG7s0PuBUJCQkSLdu3SQ3N7dsrHnz5qbHXWhoqKWxwfNQigXgMVavXi0jRoyQPn36OE3q6tevb3YX7tmzx7Q4IamDu2vbtq1plH18z8W77rrLspjguZixA+D2VqxYYWbotDzlTEhIiNx5551yxx13SFhYmEvjA6pK14Vq6fX77793GNc3MCNHjrQsLngeEjsAbmvZsmUydepU+eGHH5xeEx4ebsqtWnbV5A7wVElJSdKlSxc5evRo2ViTJk1ky5Yt0rBhQ0tjg+egFAvAreh7TZ21OOuss8yicmdJXUREhCm1aslVS68kdfB0ulN71qxZDmOpqalyyy23WBYTPA8zdgDcgv4q+vbbb80MnZ6b6YzOYGg7iBtvvNFskAC87d/BRRddJIsXL3YYX7hwoYwdO9ayuOA5SOwAWEp/BX311VcmoVuzZo3T66KiokxTYT0iLDAw0KUxAq508OBB6dy5s2RkZJSNaSlWS7L6xgY4GUqxACxL6HRhuO5w1Z2uzpI6bfvw8ssvm8bCujGCpA7eTt/EaBPt8g4fPizXX3+9+XcDnAwzdgBcvvvvs88+k2nTppkGw860bNlSHn74YZkwYYLUrVvXpTECVtNb85gxY05ovr1gwQK56qqrLIsL7o/EDoDLOux//PHHJqHTkpIzbdq0MZshrrzySvH393dpjIA7OXTokCnJ6p+ldJOQHqGnM9lARSjFAqhRdrvdnN+qN6jx48c7TepiYmLMbMSOHTtk0qRJJHWo9Ro1aiSvvfaaw1hmZqZcc801lGThFIkdgBpRWFgob731ljnoXGffNGGriH7+vffek23btpkSk81mc3msgLsaNWqU/POf/3QY093jb7zxhmUxwb1RigVQrQoKCuTtt9+WGTNmSGJiotPrtBHro48+KqNHjxY/Pz+Xxgh4Et0dq/9eDhw4UDamrX42bdpkli4A5TFjB6BaHDt2TGbPnm1KqtqSxFlS16NHD7MgXDdOaF8ukjrg5PSIvLlz5zqM5eTkmCULuhkJKI/EDkCV5OXlyUsvvWQOMr/55pvNsUgV0bYmixYtkvXr18sll1wivr78+gEq64ILLpBrr732hCP39N8eUB6lWACnJTc31yzsfuaZZyQlJcXpdf3795cpU6bI+eefLz4+Pi6NEfAmWVlZ0q1bN9m7d2/ZWL169eS3336TM844w9LY4D5I7ACckuzsbFNyffbZZ+WPP/5wel1sbKxJ6IYNG0ZCB1STH3/8Uf72t785jA0YMEB++eUXNh7BoBYCoNKzBU8++aS0atVK7r//fqdJ3dChQ+WHH34wN5pzzz2XpA6oRmeffbbcdtttDmO//vqreaMFKGbsAJzUkSNH5N///rfMmjXL4ezK4+nMnO5yPfPMM10aH1Db6MYJ3YS0a9eusrE6derIunXrpGvXrpbGBuuR2AGoUHp6uknmXnzxRTNbd7JF3ZrQDRw40KXxAbXZypUrZciQIQ67Ynv27CmrV6+muXctRykWgIO0tDRzRque1frEE084TeouuugiWbNmjXz11VckdYCLDRo0SO655x6HsQ0bNsj06dMtiwnugRk7AEZqaqo899xz8uqrr5pSz8k64esMnc4OALBOfn6+9O7dW7Zu3Vo2pn0hddZOx1E7kdgBtdzBgwdl5syZMmfOHNOTriK6AeLSSy+VRx55xLRbAOAe4uLiTEuhoqKisjE9l1nX22krFNQ+lGKBWmr//v1md13r1q3lhRdeqDCp0ybCl19+uWzevFk++ugjkjrAzejM3OTJkx3GtmzZYloNoXZixg6oZbS56VNPPSXz5s0z57pWRMs5V1xxhVlrR+NTwL3pv2PtZadr7Mq/KdOWQ7oWD7ULiR1QS+zevdv0oXvrrbfEbrdXeI02OL3qqqvkoYceknbt2rk8RgCn5/fffzezd4WFhWVjem6znkoRGBhoaWxwLUqxgJeLj4+XiRMnSvv27c1B4hUlddoe4frrr5edO3fKm2++SVIHeBjtXzd16tQT/u3rmzTULszYAV5q+/btpvXB+++/79DrqjxtaqoHiz/wwAMSHR3t8hgBVB990zZ48GCzK7Y8PQlGT6xA7UBiB3gZ3egwbdo0s9nB2T9v3S2nM3R6NFizZs1cHiOAmrFjxw5zKoW2QimlPSm1VFu/fn1LY4NrUIoFvMTGjRtNSxItySxcuLDCpC4gIMA0NU1MTDQnSpDUAd5FNzvpWtrjN0wd38wY3osZO8AL+ljpCRFffPGF02uCgoLk1ltvlbvvvlsaN27s0vgAuJYuvfjb3/4my5Ytcxj/+uuv5fzzz7csLrgGiR3goXQdjSZ0ixcvdnqNll5uv/12ufPOOyUiIsKl8QGwdhe89p0sf4pM06ZNzVKNsLAwS2NDzaIUC3iYFStWmHfd2rfKWVIXEhJiGpRqCUbX25HUAbVLmzZtzBGB5R04cEDuuOMOy2KCazBjB3gILatoOwPd4eaMvhPXcqueKKHJHYDaS2/v+ibw22+/dRj//PPP5eKLL7YsLtQsEjvAjek/T03kNKH7+eefnV6nM3L33nuv3Hzzzex8A1Bm3759ZkNVZmZm2Zius9WSbKNGjSyNDTWDUizgpgndkiVLTE+qYcOGOU3q9Bf0s88+K3v27DG96EjqAJTXokULswO+vD/++MO8CWRexzsxYwe4Ef3n+NVXX5kZujVr1ji9LioqyiRy1113HccFAfjL3ytael20aJHD+AcffCCXXXaZZXGhZpDYAW5A/xnqL11N6NavX+/0uubNm8uDDz4o11xzjWkyDACVkZKSIp07d5b09HSHNblbtmwxbxThPSjFAhb3m/rkk0+kZ8+e5h21s6ROO8fPmTNHdu3aJbfccgtJHYBTEhkZKbNnz3YYy8jIMCfQML/jXZixAyxQVFQkH3/8sWlFou+YT9ayYPLkyXLllVeKv7+/S2ME4H3GjRtnjhssb968eTJx4kTLYkL1IrEDXHxI94cffmgSOj3T0ZmYmBh55JFH5PLLLxebzebSGAF4r7S0NOnSpYukpqaWjTVo0MCcJRsdHW1pbKgelGIBFygsLJS33npLOnbsaGbfnCV1+vn33ntPtm3bJldddRVJHYBqpa2RXn/9dYexrKwss26XeR7vQGIH1KCCggKZO3euOZhbSx26Rq4i+g564cKF5l2zztL5+fm5PFYAtcPIkSNlwoQJDmNLly4163jh+SjFAjXg2LFjMn/+fHnyySclKSnJ6XU9evSQRx991Gyc8PXlfRYA1zhy5Ih5Q5mcnFw2pq2TNm3aJG3btrU0NlQNdxKgGuXl5clLL71kfjHedNNNTpO6Pn36mPYmugv2kksuIakD4FKhoaFm00R5ubm5prKgm7vgubibANVAfyG+8MILZhfr7bff7vAuuLz+/fubBsTafPiiiy4SHx8fl8cKAOq8886TG264wWHsl19+OeGkCngWSrFAFWRnZ5veUHqslx7T40xsbKxMmTLFHA9GMgfAXRw9elS6d+8uiYmJZWN169aVDRs2mM1c8DwkdsBp0F1kr7zyijz33HNy+PBhp9cNHTpUHnvsMfMnCR0Ad7Rs2TLzO6q8vn37ysqVK9mZ74EoxQKnuOD4iSeekFatWsnDDz/sNKnTmTn9Zfnjjz/K2WefTVIHwG2dddZZcueddzqMrV27Vp555hnLYsLpY8YOqAQ9X1HXnehHZmam0+suuOACs8t14MCBLo0PAKq68Ut36e/cubNsTE+70QRPS7XwHCR2wF90aX/++efl5ZdfNmtRnNGNEJrQafkCADzRr7/+atYD6xnWpTSp081ederUsTQ2VB6lWKACetzO/fffb0qu2ovOWVI3atQoiYuLM61LSOoAeLIBAwaY33vlbdy40Sw/gedgxg4o5+DBgzJz5kzTgV1LExXR9XKXXnqpOcu1W7duLo8RAGqyubr22dy8eXPZmJ6Es2rVKt68eggSO0BE9u/fbxYK6xmK+outItpEeNy4cTJ58mTp3Lmzy2MEAFfQVif9+vUTu91eNqatT7Sher169SyNDX+NUixqtb1798rNN99sTorQEyMqSur03epVV10lW7dulffff5+kDoBX69mzp1kzXN62bdtOGIN7YsYOtdLu3bvN2rm33nrL4V1pedq/SRO6hx56SNq1a+fyGAHAKoWFhWZ3v64hLr8M5eeff5bBgwdbGhtOjsQOtUp8fLzMmDFD3nnnHafnIeoWfz0v8cEHH5TWrVu7PEYAcAdbtmyRXr16SUFBQdmYVjd0Q0VQUJClscE5SrGoFbZv3y5XXnmldOjQwczSVZTU6XZ+Lcvu2rVLXnvtNZI6ALWaLjs5fkdsQkKCPPDAA5bFhL/GjB28/h3ntGnTZOHCheLsr7ouBr7++uvNNv9mzZq5PEYAcFf6JnjIkCFmV2x5S5culXPOOceyuOAciR28UmnvpU8//dTpNQEBAWaG7t5775XIyEiXxgcAnrSERRsVl28BFR0dLb///rs0aNDA0thwIkqx8Cq60Pfiiy82R+M4S+p0bYiWEvbs2SPPPvssSR0AnERMTIw8/fTTDmNJSUly9913WxYTnGPGDl5h9erVZoZu8eLFTq+pX7++3H777eaw64iICJfGBwCeTI8ZGzZsmPz4448O419++aVceOGFlsWFE5HYwaOtXLlSpk6dKkuWLHF6TUhIiEnm7rjjDgkLC3NpfADgLbTK0bVrV8nOzi4b04qHrmUODw+3NDb8D6VYeKRly5aZd496YLWzpE6TOJ3F0ybE//rXv0jqAKAK9Ozs559/3mEsJSVFbrvtNstiwomYsYPH0L+qWgZ4/PHHTZNMZ7TMqhsidGOEll8BANX3e3j48OHyzTffOIx/8sknMnr0aMviwv+Q2MHt6V/Rb7/91pRctfTqTOPGjU3LkhtvvJHmmQBQQ5KTk6VLly5y5MgRhzfUWpLV38OwFqVYuHVCp5shBgwYIOeff77TpC4qKkpmzZoliYmJcs8995DUAUAN0n6ferZ2eWlpaeZNNXNF1mPGDm5H/0ouWrTIzNCtX7/e6XXNmzc3x35dc801pskwAMB1v6e19Pr55587jL/77rtyxRVXWBYXSOzgZtvp9ZeEbnjQBsPOtGzZUh566CG5+uqrpW7dui6NEQDwX3/88Yc5dkxn60qFhobK5s2bOcXHQpRi4RZH1nz44YfSrVs3ufTSS50mdW3atJE333zTdEG/4YYbSOoAwEK6nm727NkOY7ru7rrrrqMkayESO1jGbrebaXt9xzd+/Hiz8NZZ1/MFCxbIjh07ZNKkSeLv7+/yWAEAJ9I34/r7u7yvv/7avAmHNSjFwuUKCwvlvffek+nTp8uuXbucXtexY0d55JFHZNy4ceLn5+fSGAEAlZOenm7eoGtPu1LBwcHmLFntfQfXYsYOLlNQUCBz586VM844QyZOnOg0qdNt9AsXLjS/FC6//HKSOgBwY3rqhP5uL09Pp9AKi66dhmuR2KHGHTt2TObMmWNKqrr2QtuSVKRHjx7y6aefmjV2Y8eOJaEDAA+h58VqIleeNpR/9dVXLYuptqIUixqTn59v3sU99dRTpqGlM3369JHHHntMRowYIT4+Pi6NEQBQPTIzM81Zsvv27SsbCwgIMG/W9Y09XIPEDtUuNzdXXnvtNZk5c6YcPHjQ6XX9+/eXKVOmmObDJHQA4PmWLl0q5557rsPYoEGDzDGQVGFcg1Isqo2uqdBkrnXr1nL33Xc7TepiY2PNEWGrVq2SCy64gKQOALzEsGHDzDnd5empQc8//7xlMdU2zNihyrKysuSVV16R5557Tg4fPuz0uqFDh5qSq/5JMgcA3vsmX9dMJyQklI3VqVPHnCSku2dRs0jscNq0EaWeF/jCCy9IRkbGSd/BPfroo3LmmWe6ND4AgDWWL19ufueXTzF69+5tKjX0Iq1ZlGJxWj2LdG2c9ifSGThnSZ2WWXUK/rvvviOpA4BaZPDgwWZJTnlxcXFmMx1qFjN2qDQ9D1Bn53SW7ujRo06vu+iii8wMXd++fV0aHwDAfeTl5UmvXr1k+/btZWM2m03WrFkjPXv2tDQ2b0Zih7+Umppq1s9pP6KcnByn140aNcqcFKH/kAEA0CROd8XqmeCltCXK2rVrOe+7hlCKhVO6q1Wn0nWXq+52rSip000QY8aMMX2KPvvsM5I6AECZfv36yYMPPugwpqcKPf7445bF5O2YscMJ9u/fL88884y8/vrr5tSIivj6+pozXCdPnswuJwDASY+T1KU5mzZtcriH6Bps7WeK6kVihzJJSUlmYeubb75p/iFWRBtMXnHFFfLwww+bM18BAPgrWtXR5K6wsLBsTO8hGzZsMKdToPpQioU5u/X666+Xdu3ayezZsytM6nTBq54DqItgFyxYQFIHAKi07t27m24K5e3YscNUfVC9mLGrxeLj42XGjBnyzjvvOCxsLU/7DU2cONGskdC1dgAAnA673W42UujGifLrtH/66SdaYlUjErtaSGfdpk+fLu+//74UFxdXeI12Cb/22mvlgQcekOjoaJfHCADwPtu2bTOtTsqv39ZJA11/FxwcbGls3oJSbC2yZcsWGT9+vHTq1EnefffdCpO6evXqye233y67d+82x4SR1AEAqkvHjh3NxMLxy4Huu+8+y2LyNszY1ZJFq9OmTZNPPvnE6TW6eFUPbr733nslMjLSpfEBAGoPXfqjZ4brsWPlLVmyRM477zzL4vIWJHZeTA9cfuKJJ+Q///mP02uCgoLk1ltvNf3qGjdu7NL4AAC1U0JCgnTr1k1yc3PLxpo3b2563IWGhloam6ejFOulnb71WC89cNlZUle/fn2zG2nPnj2mxQlJHQDAVdq2bWsa3x/fQ/Wuu+6yLCZvwYydF9Fmj1OnTjXT2c6EhITInXfeKXfccYeEhYW5ND4AAErpOm8tvX7//fcO41988YWMHDnSsrg8HYmdF/j5559NQnf8P47yNInTcuttt91mkjsAANyhMX6XLl3k6NGjZWNNmjQxm/0aNmxoaWyeilKsh9J8/IcffjALUM866yynSV1ERIQpte7du1ceeeQRkjoAgNvQzguzZs1yGEtNTZVbbrnFspg8HTN2HkZ/XN99952ZoVuxYoXT63TN3P333y833nij2SABAIC73td0XfjixYsdxhcuXChjx461LC5PRWLnIfTH9PXXX5uEbvXq1U6vi4qKMk2Fr7vuOgkMDHRpjAAAnI6DBw9K586dJSMjo2xMS7FaktXSLCqPUqwHJHS6kFQPT77wwgudJnW6Tfzll182jYV1YwRJHQDAU+ikhDbFL+/w4cPmHHPmn04NM3ZuvFvo888/N33otMGwMy1btpSHH35YJkyYIHXr1nVpjAAAVBdNR8aMGSOffvqpw/iCBQvkqquusiwuT0Ni54YdufWECE3odAramTZt2pg+dFdeeaX4+/u7NEYAAGrCoUOHTElW/yylm/42b95sKlP4a5Ri3YTdbpf33nvPbPu+7LLLnCZ1MTEx5t3Ljh07ZNKkSSR1AACv0ahRI3nttdccxjIzM+Waa66hJFtJtWLGTmfB0tPTzRZq/TiUkiLH8vKkuKhIfP38pG5AgDSKjDQLNPUjPDxc/Pz8XBJbYWGhvP/+++ZQ5Pj4+JMenKztSsaNG+ey2AAAsIJWo959912HMU34dM2dp9zfreLViZ3urtH1ab+vXy/5OTlSYrdLcF6ehKSni7/dLr4lJVLs4yOFNptkhodLdkCA+NhsUi8oSLr26iXdu3evsdMZCgoK5J133pEZM2aYDQ/O6Azeo48+KqNHj/b6v4wAAJTev/X+d+DAgbIxbd2lZ8m2bt3are/vVvPKxE7/IqxcvlwS4+PFPzdXopP2SVR6uoTk5Ih/UZHTxxX6+UlmUJAcDA+XpOgWUhgYKK1jYiR2yBCzY6c6HDt2TObPny9PPvmk6bjtTI8ePUxCd/HFF4uvLxVzAEDtoi2+hg8f7jCm3SEmXHml7Nm1y+3u7+7CqxI7XaemTXvXrlghwWlp0m5vkjRPSxO/4uJTfq4iX1/ZHxEhu1pGS3ZEhPSNjZXY2Fix2WynFVt+fr7MnTvXnAKRnJzs9Lo+ffrIY489JiNGjBAfH5/Tei0AALyB9mTVe6dWrAYNGiSxfftK04IC6XjgoNvc392N1yR2KSkpsnjRIsnYnywd4uMlJjnZTMVWlU7lxjdrJttjYiS8eTMZPnKkREZGVvrxubm58vrrr8szzzxjGjA6079/f5kyZYqcf/75JHQAAIhIVlaWOTazT8+e0iwsTGK2b5dmO+OlSURElROx4ire392VVyR2eg7q5wsXSuCBg9J72zZpkJtb7a+RFRgocR07Sm7TpjJq3FjTP640oVyyZIl07dpVevXqVXZ9dna2zJkzR2bOnCl//PGH0+fVdwma0A0bNoyEDgCA4+7vH779tvgnJUnHuDgJzMoy43X860jDiAjxqcH7u6fy+MROf+iffvCBNNybJP22bhXbaUzLVpbd11dWd+4k6dHRMnr8eLMBYsiQIWYnTum5dhdccIHpnv3cc89JWlqa0+caOnSoKbnqnyR0AAA4v793+HWV5B896vD5BvUbSHBwcI3c31t6cHLn0YmdzpZpJh+auEcGbtlSLaXXykzdrurSWTJatpLPvvw/Wb58ednnGjRoYDY6HDlyxOnjdWZON0WceeaZNR4rAADecH/3KS6WPw4dkqIie7mrfEzfO/9qWhtX/Of9/Uir1nLZVVd6bFnW15M3SuiaOi2/9t+61SVJndLX6b9lq/jt3SMd2rVzaEGiawGcJXU6k7dy5Ur57rvvSOoAADiF+7tWtsJCQ00y9z8lcuRIhpRU8/094OAB+WrRIhOHJ/LYxE53v+pGCV1TV5Pl14oU5ORIu19/lWbh4WaXzslcdNFFsmbNGvnqq69k4MCBLosRAABvur/XqVNHgoOCTmjyX1hQUG2vbSsult5bt0l6crKZjPFEvp7ap05bmuju15rYKHEyBYWFZlYuKCvL7M7RrdcVTdeOGjVK4uLiZNGiRdK3b1+XxggAgDfe3+s3qC82m+NRmtVdrwvJzZUzdsbLmuXLT9rNwl15ZGKnzYe1T522NHEl/ctz+PDhsr9GTXfulIjsbIk9btauWbNm8sknnzjskgUAAFW7v/uIjzkWzN8kdz4SGBhkZvKqW/vkZBPHinLr6D2FxyV2eoyIniihzYddta6ulL2wUEpK/jctrK/fYtcuad+6tYSEhJSNawPiPXv2uDQ2AAA8WWXv7zY/P7NpomlUlISGhFRLy5Pj6eu33ZskiTt3mrg8iccldno2nB4joh2nXc3m/993COVF7NsngXa7OXeuVOPGjaV58+Yujw8AAE9l5f29Ii3S0sSWmyubNm0ST+JR52cUFRWZA3/1bLjTOUakqjSli2jYUDKzskS7xPj5+ZqdOq32J8vg/v1NPx09c+6uu+6qkalhAAC8kdX394poHC337ZNNcXEyePBghy4YtXbGThdBXnHFFU4/v27dOrnvvvsq/Xzp6emSn5MjaQm7ZOSG9eajx8oV8ve4dea/n0hIqFK8KceOyS3btso569bKJb9tkNu3bZO0ggL5LDVVnkrcba7RhK1RRIQ0btRIGoY3lPCwcGmblyeh9eubr3X37t0yfvx4+fDDDyt8Dd1M8cILL5j/3r59u/To0UN69uwpq1evPqXvhTNffvmldOnSxfTT27x5c5WfDwBQO+gRXXpPKv3Iy8s75efQ4zNPR+n9PSo93WH85aS9Mnx9nIxYH2fuy/vy80/6PG/s31elx/f7dZXD/0cd/m9cGt/JzJo1yxxaUFV6atU555xjJoruvffemm1QrOedXn/99WIlTVS++vhjueinZWVboP+5aZM81rattD9uC3SRzqidwokO+m245Lff5PKoKBnz5y7XtZmZEmKzyebsbNmZmyMPtm5z4uP0B1FYKP935pny8VeLZcuWLWZcEytNbJs0aeL0NZ966inzD+l0fnj67qaidw/x8fHmczfeeKO8/PLLJskDAOCvREREnPTEpJp6Dr1nbdu27YT7+/qsLHlh7x6Z17mL+Pv6msmXAD9fCTluV+zxidmaAQOr5fGq0M9PvjzrTBk+ZsxJ76etWrUyOUplT8IoLi42ecLxjh07ZiZ6NJdISEiQZ599VmqsFPvqq6+axC4nJ0duueUW86IamCYn5557rhw9elRuvvlmUyPX8qQmFS1atJBLL73UzMz9/vvvMmHCBPMY9e2338rWrVvNdbqDVP8iTJw40Rwhojte3nrrLfONuvrqq83GBP1C9+/fL2OGDHHat+7stWtkeKNGsjwjQ+5v1VoOFRbI2wcOSGFxiQwMDZWH2/w3MfvPH6knjK/MPCKBfr5lSZ3q++eGCE3sSn13OE3m7Nsn9pISaehnk8mNG0tASbHsWL9eEhMTy67Tr/Pf//63+dr0h6cJ3BdffGH+f+fOnWZaV48d0/FvvvlGJk2aJO+88445jiw3N9ecH6tJmj7P/fffb65/8cUXZd++fWZjRufOneXxxx8/4XugyZ5+5Ofnm2sDAwMr+yMGANRier/RqlN5P//8s7mXacIRExNj7vlauXr44YdNIqOzVKNHj5brrrvOJCHaDqxTp05mxk8nGDRf0HufmjFjhrRv397kBdqof8SIEfLLL7/IAw88YPKELz76SOYdPSoDQkLkgVatJTU/X0L8bDoDZSZfIuvWLYvrl4wMeSlprxwrLpaYwECZEdNeXk5KkqN2+3+refXrS2xomITZ/E1Sp/7q8XWOS7Re379PvklLk/QtmyUhJUVee+01Mz59+nRTldNcR/MW/X7oRI72tdW8RStzej/X2UuNW3MfrcjpvVt72+r9+7fffpMNGzZIQECAw2vWrVvXfG+O/znUyIydvrhOy+oPU9t46A9GkzFNODTT1h+Mv7+/+YL1L4cmerqTpDSxu+2226Rbt27mh6/Po8mHNv8rTexuvfVWiY6ONkmMnrn63nvvmW+OJnaazes36ZGHH5bP33pL/tO2XVlc5WfsNLG7pllz+WfTprIrN1de3LtHXujQUWw+PnLfjh0m6WtRr16F4/vy82R/fr483KbtCV+7lmJLZ+wy7YUS7Osnhw4dko/TD0tuSYlcGRYml6elSa9Bg+TLr78+7R8GAAC1UYvmzeWh2FjptG6dzEhNlbODg6V7QIDckpwsxSUl0icwSEZFRkq/Ro0kvbBQ7tq+XV7r1Enq+fmZe3pD/zrm3t+v3Ixbtt0ul23aaCp4muT9o3Fj6Vq/fqUerxNEP6QflkfbtJVf27eXp1f/anKTpKQkMymjhw5oEqYlWp2MKj9jp50xNDlbu3atmVzRhO+NN96Qhg0bSrt27WT9+vUmHzoZndzS56vRGbtSOtOm67imTZtm/l9n8FJTU2Xp0qUmEVM6Q6WzbOW3COupC1OnTjV94MaOHStt/pw9K6Vnruo3Sunn77jjjrLPXXzxxebPZpGRcvi4Q4CPd0FEhPlz1ZEj8tvRo6amrvKLiqVLcLBJ3ioar2zV9kD+MZm2K96svdNMv1O9emY8Jjxc1q1fX7knAQAAZQ6np8uT33wjdfLy5FhJibSvW1cGBgXJG82by295eRKXlyc3xu+Uf9tsUqhVstwcGbtpo3lsQXGxDA0PP+E5g202+U/PXrL6yBFTlZu4ebO82KGDFFTi8cuPZMhP6RmyLmuD5G3dIrk2m6m2aa6is3Sa1ClN6o6nCZ2ukyv9nE5w6eP+8Y9/mBnLv0rqqqrSiV2HDh3Mnzob93//93/SsmXLU3qhyy+/XPr162ceq6Xbjz/++KTX6xRnqdJvoJSUmMz9ZDT7NpdKiYyNjJTboh3jfPtAcoXjKzIy5Ns0bT58ctN2J8iExo2li4+PrMzJkW/+TDSv6tFDvsjPl0XM2AEAcEo6xMTIrW3aSJvjWotoZa1PYKD5CPWzyffph2VwaJgMDQuXp9q3/8vntfn4SGxYmPkIt/nL0ko+vrhE5NboaLmkSRPZ2Ka1HB00SC655BKToFWFK5ZHVXpXrJZK1XnnnWfq7aW0TqyGDRsms2fPLkv+MjMzHR6v9eK2bduaViD6HLq+rjwt6b7//vvmv7U0q0ngCcGewlbjgSGh8tWhQ5JRWGj+/3BBgfxRUOB0fFBoqGQX2U3ZtdS6zEzZmZPj8LzZRUXSKiTUlJK/LTd7+EdurplmBQAApyZhzx45+ud9OcNul8N2uyQVFEjyn2O+Pr6SLCXStG5d6dmgvqzOPCLJf+5w1ZJr6W5XPx8fU3pVu3NzJenPnb266kyXVP3V40sNDguVj1NTJK+oSIp9fCX9yBGT12iuM3/+fLPmUJXulq1fv75ZgqY0f/n+++9N1VKv++yzz2TIkCHiKpWesbvmmmvMn48++qgpk+pUot1uN+vt3n33XTOuCyW7du1qkh5dO1e+Sa/WpvU6XYens316lqpOV5b617/+ZdbTvf3222WbJ45XNyBASipZM40JCpKbWkTLhM2/mx+oLp58Oqa90/HGderIqx07yRO7d8sr+5Kkrq+vWVCp9fXyNIO/cetWCfW3Sa/69WXfn4nfwi1bJPG4MrEukty1a5f5fuj3STdGaPKqSe2TTz5pytmaDN50001mgeqcOXPM57W8rTtl9axZ/R7rItR58+Y5XO+MbsTQJFzXP4aGhkpsbKxZrwgAwMnohkfddFeeJiiPPfaYFBYWmkrazJkzzfoxXS+/Zs0acz/XTYDXXnutDB8+XCZPnixff/21ufe89NJLZiJIu2roc+u9/fzzz5crr7zSVAF1/X3pLtJ77rpLnvv4Y6mXn282MTwVE2OmzZ7YnSDZ9iLTSLZzULBcGdXUVOamtYuR27Zvk8LiYhPX5NZtzBr6UY2bmNYmuvlRq3NTExLMhIyqzONLnRkWbtbqj934m2Rv2ybBq1bKP6++2nyNem/We7rmM1qW1ZxIvx9nn322KbXqsjTdAKnfp9LNE3p9ZU+kOuOMM8w6fv2e6yaNX3/99ZQOPaj05gl3oH/BdixZIueu+lXcSUFhgXzTp498tW2b/PDDD2XlYz08OCwszOrwAABwa+56f1ffDRwgZ/z972bdnCfwqJMntCdcXECA6Svj/2cG7g586gVIUcOGZlt306ZNTaatM24kdQAAeO79vdDPT7IDAk7ak9bdeFxi52OzSWZQkERkZYm70Hg0Lq2h6+JKV9Aav/a1K2/MmDFmGhwAAE/i7vf3JtWc2GmHkONnALXSpz17a1Vip/X5ekFBcjA83K1+8Acb/jeuirY91xSt6+sHAACerrbd3xs2bFi2+dSjzoqtbroJoWuvXpIU3UKKKjiKwwoax94WLaRb794ec0AwAADuhPt79XGP794p6N69uxQGBsr+PxsRW21fRITYAwNrvOEgAADejPt7LU3sdENC65gY2dUyWoore1xEDdHXT2gZLa3bt2ejBAAAVcD9vZYmdip2yBDJjoiQ+GbNLI1jZ7NmJo7YwYMtjQMAAG/A/b2WJnZRUVHSNzZWtsfESJYLjueoSGZgoOxoHyP9Bg828QAAgKrh/l5LEzulXa3DmjeTuI4dxe7ihZb6enGdOkp4s2YyaNAgl742AADejPt7LU3s9AiTC0eOlNymTWV1504uq8fr6+jr5UU1leEjR5o4AABA9eD+XksTOxUZGSmjxo2V9OhoWdWlc41n9vr8+jr6evq6+voAAKB6cX8/fR51Vqwze/fulc8XfiSBBw5I723bpEFubo3U3HV6VjN5/aHrwccAAKDmcH+vpYmdSklJkcWLFknG/mTpEB8vMcnJ4lsNX5pOzeruGF1IqTV3nZ715EweAABPwv29liZ2ym63y4oVK2TtihUSnJYmbfcmSYu0NPErLj6tjtPanFD72OiWZ90dowspPbXmDgCAp+L+XksTu1IHDhyQlStWSOLOnWLLzZWW+/ZJ1OF0CcnJEf+iIqePK/TzMwf+6tlweoyIdpzW5oSxHrrlGQAAb8L9vZYmdqUyMjJk06ZNsikuTvJzcqTEbpfgvDxpkJ4hdex28S0plmIfXymw2SQrPEyyAwLEx2YzB/7q2XB6jIindZwGAMDbcX+vpYldqaKiIklPT5fU1FTzcSglRQry86XIbhc/m03q1KsnjSIjpUmTJuYjPDzcow78BQCgNuL+XksTOwAAgNrAo/vYAQAA4H9I7AAAALwEiR0AAICXILEDAADwEiR2AAAAXoLEDgAAwEuQ2AEAAHgJEjsAAAAvQWIHAADgJUjsAAAAvASJHQAAgJcgsQMAAPASJHYAAABegsQOAADAS5DYAQAAeAkSOwAAAC9BYgcAAOAlSOwAAAC8BIkdAACAlyCxAwAA8BIkdgAAAF6CxA4AAMBLkNgBAAB4CRI7AAAAL0FiBwAA4CVI7AAAALwEiR0AAIB4h/8HAPSWz2Mn9rYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "graph_search_space = tpot.search_spaces.pipelines.GraphSearchPipeline(\n",
    "    leaf_search_space = fss_search_space,\n",
    "    inner_search_space = tpot.config.get_search_space([\"transformers\"]),\n",
    "    root_search_space= tpot.config.get_search_space([\"KNeighborsClassifier\", \"LogisticRegression\", \"DecisionTreeClassifier\"]),\n",
    "    max_size = 10,\n",
    ")\n",
    "\n",
    "graph_search_space.generate(rng=4).export_pipeline().plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Optimize with TPOT\n",
    "\n",
    "For this example, we will optimize the DynamicUnion search space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ketrong/Desktop/tpotvalidation/tpot/tpot/tpot_estimator/estimator.py:456: UserWarning: Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\n",
      "  warnings.warn(\"Both generations and max_time_mins are set. TPOT will terminate when the first condition is met.\")\n",
      "Generation: 100%|██████████| 5/5 [00:41<00:00,  8.33s/it]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9838836477987423\n"
     ]
    }
   ],
   "source": [
    "import tpot\n",
    "import sklearn.datasets\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "final_classification_search_space = SequentialPipeline([dynamic_fss_space, classification_search_space])\n",
    "\n",
    "est = tpot.TPOTEstimator(generations=5, \n",
    "                            scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n",
    "                            scorers_weights=[1.0, -1.0],\n",
    "                            n_jobs=32,\n",
    "                            classification=True,\n",
    "                            search_space = final_classification_search_space,\n",
    "                            verbose=1,\n",
    "                            )\n",
    "\n",
    "\n",
    "scorer = sklearn.metrics.get_scorer('roc_auc_ovr')\n",
    "\n",
    "est.fit(X_train, y_train)\n",
    "print(scorer(est, X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that this pipeline performed slightly better and correctly identified group one and group two as the feature sets used in the generative equation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-8 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-8 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-8 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-8 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-8 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-8 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-8 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-8 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-8 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-8 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-8 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-8 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-8 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-8 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-8 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-8 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-8 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-8 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-8\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                                 FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                                                    sel_subset=[&#x27;d&#x27;,\n",
       "                                                                                &#x27;e&#x27;,\n",
       "                                                                                &#x27;f&#x27;])),\n",
       "                                                (&#x27;featuresetselector-2&#x27;,\n",
       "                                                 FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                                    sel_subset=[&#x27;a&#x27;,\n",
       "                                                                                &#x27;b&#x27;,\n",
       "                                                                                &#x27;c&#x27;]))])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(max_features=0.0530704381152,\n",
       "                                        min_samples_leaf=2, min_samples_split=5,\n",
       "                                        n_estimators=128, n_jobs=1))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-15\" type=\"checkbox\" ><label for=\"sk-estimator-id-15\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;&nbsp;Pipeline<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for Pipeline</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></label><div class=\"sk-toggleable__content fitted\"><pre>Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                                 FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                                                    sel_subset=[&#x27;d&#x27;,\n",
       "                                                                                &#x27;e&#x27;,\n",
       "                                                                                &#x27;f&#x27;])),\n",
       "                                                (&#x27;featuresetselector-2&#x27;,\n",
       "                                                 FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                                    sel_subset=[&#x27;a&#x27;,\n",
       "                                                                                &#x27;b&#x27;,\n",
       "                                                                                &#x27;c&#x27;]))])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(max_features=0.0530704381152,\n",
       "                                        min_samples_leaf=2, min_samples_split=5,\n",
       "                                        n_estimators=128, n_jobs=1))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-16\" type=\"checkbox\" ><label for=\"sk-estimator-id-16\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;featureunion: FeatureUnion<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion: FeatureUnion</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>FeatureUnion(transformer_list=[(&#x27;featuresetselector-1&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                                   sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])),\n",
       "                               (&#x27;featuresetselector-2&#x27;,\n",
       "                                FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                   sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;,\n",
       "                                                               &#x27;c&#x27;]))])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><label>featuresetselector-1</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-17\" type=\"checkbox\" ><label for=\"sk-estimator-id-17\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">FeatureSetSelector</label><div class=\"sk-toggleable__content fitted\"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><label>featuresetselector-2</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-18\" type=\"checkbox\" ><label for=\"sk-estimator-id-18\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">FeatureSetSelector</label><div class=\"sk-toggleable__content fitted\"><pre>FeatureSetSelector(name=&#x27;group_one&#x27;, sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-19\" type=\"checkbox\" ><label for=\"sk-estimator-id-19\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow fitted\">&nbsp;RandomForestClassifier<a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.RandomForestClassifier.html\">?<span>Documentation for RandomForestClassifier</span></a></label><div class=\"sk-toggleable__content fitted\"><pre>RandomForestClassifier(max_features=0.0530704381152, min_samples_leaf=2,\n",
       "                       min_samples_split=5, n_estimators=128, n_jobs=1)</pre></div> </div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('featureunion',\n",
       "                 FeatureUnion(transformer_list=[('featuresetselector-1',\n",
       "                                                 FeatureSetSelector(name='group_two',\n",
       "                                                                    sel_subset=['d',\n",
       "                                                                                'e',\n",
       "                                                                                'f'])),\n",
       "                                                ('featuresetselector-2',\n",
       "                                                 FeatureSetSelector(name='group_one',\n",
       "                                                                    sel_subset=['a',\n",
       "                                                                                'b',\n",
       "                                                                                'c']))])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(max_features=0.0530704381152,\n",
       "                                        min_samples_leaf=2, min_samples_split=5,\n",
       "                                        n_estimators=128, n_jobs=1))])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "est.fitted_pipeline_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combining with existing search spaces\n",
    "\n",
    "As with all search spaces, FSSNode can be combined with any other search space. \n",
    "\n",
    "You can also pair this with the existing prebuilt templates, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-9 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-9 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-9 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-9 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-9 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-9 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-9 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-9 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-9 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-9 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-9 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-9 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-9 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-9 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-9 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-9 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-9 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-9 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-9\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                 FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                    sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])),\n",
       "                (&#x27;pipeline&#x27;,\n",
       "                 Pipeline(steps=[(&#x27;maxabsscaler&#x27;, MaxAbsScaler()),\n",
       "                                 (&#x27;rfe&#x27;,\n",
       "                                  RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                                                     min_samples_leaf=8,\n",
       "                                                                     min_samples_split=14,\n",
       "                                                                     n_jobs=1),\n",
       "                                      step=0.753983388654)),\n",
       "                                 (&#x27;featureunion-1&#x27;,\n",
       "                                  FeatureUnion(transformer_lis...\n",
       "                                  FeatureUnion(transformer_list=[(&#x27;skiptransformer&#x27;,\n",
       "                                                                  SkipTransformer()),\n",
       "                                                                 (&#x27;passthrough&#x27;,\n",
       "                                                                  Passthrough())])),\n",
       "                                 (&#x27;histgradientboostingclassifier&#x27;,\n",
       "                                  HistGradientBoostingClassifier(early_stopping=True,\n",
       "                                                                 l2_regularization=9.1304e-09,\n",
       "                                                                 learning_rate=0.0036310282582,\n",
       "                                                                 max_features=0.238877814721,\n",
       "                                                                 max_leaf_nodes=1696,\n",
       "                                                                 min_samples_leaf=59,\n",
       "                                                                 n_iter_no_change=14,\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 validation_fraction=None))]))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-20\" type=\"checkbox\" ><label for=\"sk-estimator-id-20\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;&nbsp;Pipeline<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for Pipeline</span></a><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                 FeatureSetSelector(name=&#x27;group_two&#x27;,\n",
       "                                    sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])),\n",
       "                (&#x27;pipeline&#x27;,\n",
       "                 Pipeline(steps=[(&#x27;maxabsscaler&#x27;, MaxAbsScaler()),\n",
       "                                 (&#x27;rfe&#x27;,\n",
       "                                  RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                                                     min_samples_leaf=8,\n",
       "                                                                     min_samples_split=14,\n",
       "                                                                     n_jobs=1),\n",
       "                                      step=0.753983388654)),\n",
       "                                 (&#x27;featureunion-1&#x27;,\n",
       "                                  FeatureUnion(transformer_lis...\n",
       "                                  FeatureUnion(transformer_list=[(&#x27;skiptransformer&#x27;,\n",
       "                                                                  SkipTransformer()),\n",
       "                                                                 (&#x27;passthrough&#x27;,\n",
       "                                                                  Passthrough())])),\n",
       "                                 (&#x27;histgradientboostingclassifier&#x27;,\n",
       "                                  HistGradientBoostingClassifier(early_stopping=True,\n",
       "                                                                 l2_regularization=9.1304e-09,\n",
       "                                                                 learning_rate=0.0036310282582,\n",
       "                                                                 max_features=0.238877814721,\n",
       "                                                                 max_leaf_nodes=1696,\n",
       "                                                                 min_samples_leaf=59,\n",
       "                                                                 n_iter_no_change=14,\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 validation_fraction=None))]))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-21\" type=\"checkbox\" ><label for=\"sk-estimator-id-21\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_two&#x27;, sel_subset=[&#x27;d&#x27;, &#x27;e&#x27;, &#x27;f&#x27;])</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-22\" type=\"checkbox\" ><label for=\"sk-estimator-id-22\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;pipeline: Pipeline<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for pipeline: Pipeline</span></a></label><div class=\"sk-toggleable__content \"><pre>Pipeline(steps=[(&#x27;maxabsscaler&#x27;, MaxAbsScaler()),\n",
       "                (&#x27;rfe&#x27;,\n",
       "                 RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                                    min_samples_leaf=8,\n",
       "                                                    min_samples_split=14,\n",
       "                                                    n_jobs=1),\n",
       "                     step=0.753983388654)),\n",
       "                (&#x27;featureunion-1&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                 FeatureUnion(transformer_list=[(&#x27;columnordinalencoder&#x27;,\n",
       "                                                                                 ColumnOrdinalEncoder()),\n",
       "                                                                                (&#x27;pca&#x27;,\n",
       "                                                                                 PCA(n_co...\n",
       "                 FeatureUnion(transformer_list=[(&#x27;skiptransformer&#x27;,\n",
       "                                                 SkipTransformer()),\n",
       "                                                (&#x27;passthrough&#x27;,\n",
       "                                                 Passthrough())])),\n",
       "                (&#x27;histgradientboostingclassifier&#x27;,\n",
       "                 HistGradientBoostingClassifier(early_stopping=True,\n",
       "                                                l2_regularization=9.1304e-09,\n",
       "                                                learning_rate=0.0036310282582,\n",
       "                                                max_features=0.238877814721,\n",
       "                                                max_leaf_nodes=1696,\n",
       "                                                min_samples_leaf=59,\n",
       "                                                n_iter_no_change=14, tol=0.0001,\n",
       "                                                validation_fraction=None))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-23\" type=\"checkbox\" ><label for=\"sk-estimator-id-23\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;MaxAbsScaler<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.MaxAbsScaler.html\">?<span>Documentation for MaxAbsScaler</span></a></label><div class=\"sk-toggleable__content \"><pre>MaxAbsScaler()</pre></div> </div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-24\" type=\"checkbox\" ><label for=\"sk-estimator-id-24\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;rfe: RFE<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.feature_selection.RFE.html\">?<span>Documentation for rfe: RFE</span></a></label><div class=\"sk-toggleable__content \"><pre>RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                   min_samples_leaf=8, min_samples_split=14,\n",
       "                                   n_jobs=1),\n",
       "    step=0.753983388654)</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-25\" type=\"checkbox\" ><label for=\"sk-estimator-id-25\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">estimator: ExtraTreesClassifier</label><div class=\"sk-toggleable__content \"><pre>ExtraTreesClassifier(max_features=0.0390676831531, min_samples_leaf=8,\n",
       "                     min_samples_split=14, n_jobs=1)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-26\" type=\"checkbox\" ><label for=\"sk-estimator-id-26\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;ExtraTreesClassifier<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html\">?<span>Documentation for ExtraTreesClassifier</span></a></label><div class=\"sk-toggleable__content \"><pre>ExtraTreesClassifier(max_features=0.0390676831531, min_samples_leaf=8,\n",
       "                     min_samples_split=14, n_jobs=1)</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-27\" type=\"checkbox\" ><label for=\"sk-estimator-id-27\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion-1: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion-1: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                FeatureUnion(transformer_list=[(&#x27;columnordinalencoder&#x27;,\n",
       "                                                                ColumnOrdinalEncoder()),\n",
       "                                                               (&#x27;pca&#x27;,\n",
       "                                                                PCA(n_components=0.9286371732844))])),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureunion</label></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>columnordinalencoder</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-28\" type=\"checkbox\" ><label for=\"sk-estimator-id-28\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">ColumnOrdinalEncoder</label><div class=\"sk-toggleable__content \"><pre>ColumnOrdinalEncoder()</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>pca</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-29\" type=\"checkbox\" ><label for=\"sk-estimator-id-29\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;PCA<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.decomposition.PCA.html\">?<span>Documentation for PCA</span></a></label><div class=\"sk-toggleable__content \"><pre>PCA(n_components=0.9286371732844)</pre></div> </div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-30\" type=\"checkbox\" ><label for=\"sk-estimator-id-30\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-31\" type=\"checkbox\" ><label for=\"sk-estimator-id-31\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion-2: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion-2: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;skiptransformer&#x27;, SkipTransformer()),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>skiptransformer</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-32\" type=\"checkbox\" ><label for=\"sk-estimator-id-32\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">SkipTransformer</label><div class=\"sk-toggleable__content \"><pre>SkipTransformer()</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-33\" type=\"checkbox\" ><label for=\"sk-estimator-id-33\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-34\" type=\"checkbox\" ><label for=\"sk-estimator-id-34\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;HistGradientBoostingClassifier<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html\">?<span>Documentation for HistGradientBoostingClassifier</span></a></label><div class=\"sk-toggleable__content \"><pre>HistGradientBoostingClassifier(early_stopping=True,\n",
       "                               l2_regularization=9.1304e-09,\n",
       "                               learning_rate=0.0036310282582,\n",
       "                               max_features=0.238877814721, max_leaf_nodes=1696,\n",
       "                               min_samples_leaf=59, n_iter_no_change=14,\n",
       "                               tol=0.0001, validation_fraction=None)</pre></div> </div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('featuresetselector',\n",
       "                 FeatureSetSelector(name='group_two',\n",
       "                                    sel_subset=['d', 'e', 'f'])),\n",
       "                ('pipeline',\n",
       "                 Pipeline(steps=[('maxabsscaler', MaxAbsScaler()),\n",
       "                                 ('rfe',\n",
       "                                  RFE(estimator=ExtraTreesClassifier(max_features=0.0390676831531,\n",
       "                                                                     min_samples_leaf=8,\n",
       "                                                                     min_samples_split=14,\n",
       "                                                                     n_jobs=1),\n",
       "                                      step=0.753983388654)),\n",
       "                                 ('featureunion-1',\n",
       "                                  FeatureUnion(transformer_lis...\n",
       "                                  FeatureUnion(transformer_list=[('skiptransformer',\n",
       "                                                                  SkipTransformer()),\n",
       "                                                                 ('passthrough',\n",
       "                                                                  Passthrough())])),\n",
       "                                 ('histgradientboostingclassifier',\n",
       "                                  HistGradientBoostingClassifier(early_stopping=True,\n",
       "                                                                 l2_regularization=9.1304e-09,\n",
       "                                                                 learning_rate=0.0036310282582,\n",
       "                                                                 max_features=0.238877814721,\n",
       "                                                                 max_leaf_nodes=1696,\n",
       "                                                                 min_samples_leaf=59,\n",
       "                                                                 n_iter_no_change=14,\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 validation_fraction=None))]))])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linear_search_space = tpot.config.template_search_spaces.get_template_search_spaces(\"linear\", classification=True)\n",
    "fss_and_linear_search_space = SequentialPipeline([fss_search_space, linear_search_space])\n",
    "\n",
    "# est = tpot.TPOTEstimator(  \n",
    "#                            population_size=32,\n",
    "#                            generations=10, \n",
    "#                            scorers=[\"roc_auc_ovr\", tpot.objectives.complexity_scorer],\n",
    "#                            scorers_weights=[1.0, -1.0],\n",
    "#                            other_objective_functions=[number_of_selected_features],\n",
    "#                            other_objective_functions_weights = [-1],\n",
    "#                            objective_function_names = [\"Number of selected features\"],\n",
    "\n",
    "#                            n_jobs=32,\n",
    "#                            classification=True,\n",
    "#                            search_space = fss_and_linear_search_space,\n",
    "#                            verbose=2,\n",
    "#                             )\n",
    "\n",
    "fss_and_linear_search_space.generate(rng=1).export_pipeline()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting Fancy\n",
    "\n",
    "If you want to get fancy, you can combine more search spaces in order to set up unique preprocessing pipelines per feature set. Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "dynamic_transformers = DynamicUnionPipeline(get_search_space(\"all_transformers\"), max_estimators=4)\n",
    "dynamic_transformers_with_passthrough = tpot.search_spaces.pipelines.UnionPipeline([\n",
    "    dynamic_transformers,\n",
    "    tpot.config.get_search_space(\"Passthrough\")],\n",
    "    )\n",
    "multi_step_engineering = DynamicLinearPipeline(dynamic_transformers_with_passthrough, max_length=4)\n",
    "fss_engineering_search_space = SequentialPipeline([fss_search_space, multi_step_engineering])\n",
    "union_fss_engineering_search_space = DynamicUnionPipeline(fss_engineering_search_space)\n",
    "\n",
    "final_fancy_search_space = SequentialPipeline([union_fss_engineering_search_space, classification_search_space])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-10 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: black;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-10 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-10 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-10 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-10 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: block;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-10 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-10 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-10 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-10 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-10 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-10 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-10 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-10 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-10 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-10 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 1ex;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-10 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-10 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-10 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-10\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;pipeline-1&#x27;,\n",
       "                                                 Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                                                                  FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                                                     sel_subset=[&#x27;a&#x27;,\n",
       "                                                                                                 &#x27;b&#x27;,\n",
       "                                                                                                 &#x27;c&#x27;])),\n",
       "                                                                 (&#x27;pipeline&#x27;,\n",
       "                                                                  Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                                                                                   FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                                                                                   FeatureUnion(transformer_list=[(&#x27;zerocount&#x27;,\n",
       "                                                                                                                                                   ZeroCount())])),\n",
       "                                                                                                                  (&#x27;passthrough&#x27;,\n",
       "                                                                                                                   Passth...\n",
       "                                                                                                                                                   KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                                                                                                    n_bins=11)),\n",
       "                                                                                                                                                  (&#x27;rbfsampler&#x27;,\n",
       "                                                                                                                                                   RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                                                                                              n_components=17)),\n",
       "                                                                                                                                                  (&#x27;maxabsscaler&#x27;,\n",
       "                                                                                                                                                   MaxAbsScaler())])),\n",
       "                                                                                                                  (&#x27;passthrough&#x27;,\n",
       "                                                                                                                   Passthrough())]))]))]))])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(bootstrap=False,\n",
       "                                        class_weight=&#x27;balanced&#x27;,\n",
       "                                        max_features=0.8205760841606,\n",
       "                                        min_samples_leaf=16,\n",
       "                                        min_samples_split=11, n_estimators=128,\n",
       "                                        n_jobs=1))])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-35\" type=\"checkbox\" ><label for=\"sk-estimator-id-35\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;&nbsp;Pipeline<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for Pipeline</span></a><span class=\"sk-estimator-doc-link \">i<span>Not fitted</span></span></label><div class=\"sk-toggleable__content \"><pre>Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;pipeline-1&#x27;,\n",
       "                                                 Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                                                                  FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                                                     sel_subset=[&#x27;a&#x27;,\n",
       "                                                                                                 &#x27;b&#x27;,\n",
       "                                                                                                 &#x27;c&#x27;])),\n",
       "                                                                 (&#x27;pipeline&#x27;,\n",
       "                                                                  Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                                                                                   FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                                                                                   FeatureUnion(transformer_list=[(&#x27;zerocount&#x27;,\n",
       "                                                                                                                                                   ZeroCount())])),\n",
       "                                                                                                                  (&#x27;passthrough&#x27;,\n",
       "                                                                                                                   Passth...\n",
       "                                                                                                                                                   KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                                                                                                    n_bins=11)),\n",
       "                                                                                                                                                  (&#x27;rbfsampler&#x27;,\n",
       "                                                                                                                                                   RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                                                                                              n_components=17)),\n",
       "                                                                                                                                                  (&#x27;maxabsscaler&#x27;,\n",
       "                                                                                                                                                   MaxAbsScaler())])),\n",
       "                                                                                                                  (&#x27;passthrough&#x27;,\n",
       "                                                                                                                   Passthrough())]))]))]))])),\n",
       "                (&#x27;randomforestclassifier&#x27;,\n",
       "                 RandomForestClassifier(bootstrap=False,\n",
       "                                        class_weight=&#x27;balanced&#x27;,\n",
       "                                        max_features=0.8205760841606,\n",
       "                                        min_samples_leaf=16,\n",
       "                                        min_samples_split=11, n_estimators=128,\n",
       "                                        n_jobs=1))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-36\" type=\"checkbox\" ><label for=\"sk-estimator-id-36\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;pipeline-1&#x27;,\n",
       "                                Pipeline(steps=[(&#x27;featuresetselector&#x27;,\n",
       "                                                 FeatureSetSelector(name=&#x27;group_one&#x27;,\n",
       "                                                                    sel_subset=[&#x27;a&#x27;,\n",
       "                                                                                &#x27;b&#x27;,\n",
       "                                                                                &#x27;c&#x27;])),\n",
       "                                                (&#x27;pipeline&#x27;,\n",
       "                                                 Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                                                                  FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                                                                  FeatureUnion(transformer_list=[(&#x27;zerocount&#x27;,\n",
       "                                                                                                                                  ZeroCount())])),\n",
       "                                                                                                 (&#x27;passthrough&#x27;,\n",
       "                                                                                                  Passthrough())]))]))])),\n",
       "                               (&#x27;pipeline-2&#x27;,...\n",
       "                                                                                                                                  PCA(n_components=0.9470333477868))])),\n",
       "                                                                                                 (&#x27;passthrough&#x27;,\n",
       "                                                                                                  Passthrough())])),\n",
       "                                                                 (&#x27;featureunion-3&#x27;,\n",
       "                                                                  FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                                                                  FeatureUnion(transformer_list=[(&#x27;kbinsdiscretizer&#x27;,\n",
       "                                                                                                                                  KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                                                                                   n_bins=11)),\n",
       "                                                                                                                                 (&#x27;rbfsampler&#x27;,\n",
       "                                                                                                                                  RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                                                                             n_components=17)),\n",
       "                                                                                                                                 (&#x27;maxabsscaler&#x27;,\n",
       "                                                                                                                                  MaxAbsScaler())])),\n",
       "                                                                                                 (&#x27;passthrough&#x27;,\n",
       "                                                                                                  Passthrough())]))]))]))])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>pipeline-1</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-37\" type=\"checkbox\" ><label for=\"sk-estimator-id-37\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_one&#x27;, sel_subset=[&#x27;a&#x27;, &#x27;b&#x27;, &#x27;c&#x27;])</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-38\" type=\"checkbox\" ><label for=\"sk-estimator-id-38\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;pipeline: Pipeline<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for pipeline: Pipeline</span></a></label><div class=\"sk-toggleable__content \"><pre>Pipeline(steps=[(&#x27;featureunion&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                 FeatureUnion(transformer_list=[(&#x27;zerocount&#x27;,\n",
       "                                                                                 ZeroCount())])),\n",
       "                                                (&#x27;passthrough&#x27;,\n",
       "                                                 Passthrough())]))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-39\" type=\"checkbox\" ><label for=\"sk-estimator-id-39\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                FeatureUnion(transformer_list=[(&#x27;zerocount&#x27;,\n",
       "                                                                ZeroCount())])),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureunion</label></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>zerocount</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-40\" type=\"checkbox\" ><label for=\"sk-estimator-id-40\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">ZeroCount</label><div class=\"sk-toggleable__content \"><pre>ZeroCount()</pre></div> </div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-41\" type=\"checkbox\" ><label for=\"sk-estimator-id-41\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>pipeline-2</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-42\" type=\"checkbox\" ><label for=\"sk-estimator-id-42\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">FeatureSetSelector</label><div class=\"sk-toggleable__content \"><pre>FeatureSetSelector(name=&#x27;group_four&#x27;, sel_subset=[&#x27;j&#x27;, &#x27;k&#x27;, &#x27;l&#x27;])</pre></div> </div></div><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-43\" type=\"checkbox\" ><label for=\"sk-estimator-id-43\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;pipeline: Pipeline<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.Pipeline.html\">?<span>Documentation for pipeline: Pipeline</span></a></label><div class=\"sk-toggleable__content \"><pre>Pipeline(steps=[(&#x27;featureunion-1&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                 FeatureUnion(transformer_list=[(&#x27;kbinsdiscretizer&#x27;,\n",
       "                                                                                 KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                                  n_bins=37,\n",
       "                                                                                                  strategy=&#x27;kmeans&#x27;)),\n",
       "                                                                                (&#x27;featureagglomeration&#x27;,\n",
       "                                                                                 FeatureAgglomeration(n_clusters=31))])),\n",
       "                                                (&#x27;passthrough&#x27;,\n",
       "                                                 Passthrough())])),\n",
       "                (&#x27;featureunion-2&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;f...\n",
       "                                                                                 PCA(n_components=0.9470333477868))])),\n",
       "                                                (&#x27;passthrough&#x27;,\n",
       "                                                 Passthrough())])),\n",
       "                (&#x27;featureunion-3&#x27;,\n",
       "                 FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                                 FeatureUnion(transformer_list=[(&#x27;kbinsdiscretizer&#x27;,\n",
       "                                                                                 KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                                  n_bins=11)),\n",
       "                                                                                (&#x27;rbfsampler&#x27;,\n",
       "                                                                                 RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                            n_components=17)),\n",
       "                                                                                (&#x27;maxabsscaler&#x27;,\n",
       "                                                                                 MaxAbsScaler())])),\n",
       "                                                (&#x27;passthrough&#x27;,\n",
       "                                                 Passthrough())]))])</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-44\" type=\"checkbox\" ><label for=\"sk-estimator-id-44\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion-1: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion-1: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                FeatureUnion(transformer_list=[(&#x27;kbinsdiscretizer&#x27;,\n",
       "                                                                KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                 n_bins=37,\n",
       "                                                                                 strategy=&#x27;kmeans&#x27;)),\n",
       "                                                               (&#x27;featureagglomeration&#x27;,\n",
       "                                                                FeatureAgglomeration(n_clusters=31))])),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureunion</label></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>kbinsdiscretizer</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-45\" type=\"checkbox\" ><label for=\"sk-estimator-id-45\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;KBinsDiscretizer<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html\">?<span>Documentation for KBinsDiscretizer</span></a></label><div class=\"sk-toggleable__content \"><pre>KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;, n_bins=37, strategy=&#x27;kmeans&#x27;)</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureagglomeration</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-46\" type=\"checkbox\" ><label for=\"sk-estimator-id-46\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;FeatureAgglomeration<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.cluster.FeatureAgglomeration.html\">?<span>Documentation for FeatureAgglomeration</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureAgglomeration(n_clusters=31)</pre></div> </div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-47\" type=\"checkbox\" ><label for=\"sk-estimator-id-47\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-48\" type=\"checkbox\" ><label for=\"sk-estimator-id-48\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion-2: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion-2: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                FeatureUnion(transformer_list=[(&#x27;quantiletransformer&#x27;,\n",
       "                                                                QuantileTransformer(n_quantiles=840,\n",
       "                                                                                    output_distribution=&#x27;normal&#x27;)),\n",
       "                                                               (&#x27;pca&#x27;,\n",
       "                                                                PCA(n_components=0.9470333477868))])),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureunion</label></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>quantiletransformer</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-49\" type=\"checkbox\" ><label for=\"sk-estimator-id-49\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;QuantileTransformer<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.QuantileTransformer.html\">?<span>Documentation for QuantileTransformer</span></a></label><div class=\"sk-toggleable__content \"><pre>QuantileTransformer(n_quantiles=840, output_distribution=&#x27;normal&#x27;)</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>pca</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-50\" type=\"checkbox\" ><label for=\"sk-estimator-id-50\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;PCA<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.decomposition.PCA.html\">?<span>Documentation for PCA</span></a></label><div class=\"sk-toggleable__content \"><pre>PCA(n_components=0.9470333477868)</pre></div> </div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-51\" type=\"checkbox\" ><label for=\"sk-estimator-id-51\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-52\" type=\"checkbox\" ><label for=\"sk-estimator-id-52\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;featureunion-3: FeatureUnion<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.pipeline.FeatureUnion.html\">?<span>Documentation for featureunion-3: FeatureUnion</span></a></label><div class=\"sk-toggleable__content \"><pre>FeatureUnion(transformer_list=[(&#x27;featureunion&#x27;,\n",
       "                                FeatureUnion(transformer_list=[(&#x27;kbinsdiscretizer&#x27;,\n",
       "                                                                KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;,\n",
       "                                                                                 n_bins=11)),\n",
       "                                                               (&#x27;rbfsampler&#x27;,\n",
       "                                                                RBFSampler(gamma=0.0925899621466,\n",
       "                                                                           n_components=17)),\n",
       "                                                               (&#x27;maxabsscaler&#x27;,\n",
       "                                                                MaxAbsScaler())])),\n",
       "                               (&#x27;passthrough&#x27;, Passthrough())])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>featureunion</label></div></div><div class=\"sk-serial\"><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>kbinsdiscretizer</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-53\" type=\"checkbox\" ><label for=\"sk-estimator-id-53\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;KBinsDiscretizer<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.KBinsDiscretizer.html\">?<span>Documentation for KBinsDiscretizer</span></a></label><div class=\"sk-toggleable__content \"><pre>KBinsDiscretizer(encode=&#x27;onehot-dense&#x27;, n_bins=11)</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>rbfsampler</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-54\" type=\"checkbox\" ><label for=\"sk-estimator-id-54\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;RBFSampler<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.kernel_approximation.RBFSampler.html\">?<span>Documentation for RBFSampler</span></a></label><div class=\"sk-toggleable__content \"><pre>RBFSampler(gamma=0.0925899621466, n_components=17)</pre></div> </div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>maxabsscaler</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-55\" type=\"checkbox\" ><label for=\"sk-estimator-id-55\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;MaxAbsScaler<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.preprocessing.MaxAbsScaler.html\">?<span>Documentation for MaxAbsScaler</span></a></label><div class=\"sk-toggleable__content \"><pre>MaxAbsScaler()</pre></div> </div></div></div></div></div></div></div></div></div></div><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label  sk-toggleable\"><label>passthrough</label></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-56\" type=\"checkbox\" ><label for=\"sk-estimator-id-56\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">Passthrough</label><div class=\"sk-toggleable__content \"><pre>Passthrough()</pre></div> </div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator  sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-57\" type=\"checkbox\" ><label for=\"sk-estimator-id-57\" class=\"sk-toggleable__label  sk-toggleable__label-arrow \">&nbsp;RandomForestClassifier<a class=\"sk-estimator-doc-link \" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.5/modules/generated/sklearn.ensemble.RandomForestClassifier.html\">?<span>Documentation for RandomForestClassifier</span></a></label><div class=\"sk-toggleable__content \"><pre>RandomForestClassifier(bootstrap=False, class_weight=&#x27;balanced&#x27;,\n",
       "                       max_features=0.8205760841606, min_samples_leaf=16,\n",
       "                       min_samples_split=11, n_estimators=128, n_jobs=1)</pre></div> </div></div></div></div></div></div>"
      ],
      "text/plain": [
       "Pipeline(steps=[('featureunion',\n",
       "                 FeatureUnion(transformer_list=[('pipeline-1',\n",
       "                                                 Pipeline(steps=[('featuresetselector',\n",
       "                                                                  FeatureSetSelector(name='group_one',\n",
       "                                                                                     sel_subset=['a',\n",
       "                                                                                                 'b',\n",
       "                                                                                                 'c'])),\n",
       "                                                                 ('pipeline',\n",
       "                                                                  Pipeline(steps=[('featureunion',\n",
       "                                                                                   FeatureUnion(transformer_list=[('featureunion',\n",
       "                                                                                                                   FeatureUnion(transformer_list=[('zerocount',\n",
       "                                                                                                                                                   ZeroCount())])),\n",
       "                                                                                                                  ('passthrough',\n",
       "                                                                                                                   Passth...\n",
       "                                                                                                                                                   KBinsDiscretizer(encode='onehot-dense',\n",
       "                                                                                                                                                                    n_bins=11)),\n",
       "                                                                                                                                                  ('rbfsampler',\n",
       "                                                                                                                                                   RBFSampler(gamma=0.0925899621466,\n",
       "                                                                                                                                                              n_components=17)),\n",
       "                                                                                                                                                  ('maxabsscaler',\n",
       "                                                                                                                                                   MaxAbsScaler())])),\n",
       "                                                                                                                  ('passthrough',\n",
       "                                                                                                                   Passthrough())]))]))]))])),\n",
       "                ('randomforestclassifier',\n",
       "                 RandomForestClassifier(bootstrap=False,\n",
       "                                        class_weight='balanced',\n",
       "                                        max_features=0.8205760841606,\n",
       "                                        min_samples_leaf=16,\n",
       "                                        min_samples_split=11, n_estimators=128,\n",
       "                                        n_jobs=1))])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_fancy_search_space.generate(rng=3).export_pipeline()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Other examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.315442</td>\n",
       "      <td>-1.039258</td>\n",
       "      <td>0.194516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>-1.908995</td>\n",
       "      <td>-0.953551</td>\n",
       "      <td>-1.430472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>0.181162</td>\n",
       "      <td>1.022858</td>\n",
       "      <td>-2.289700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2.825765</td>\n",
       "      <td>-1.205520</td>\n",
       "      <td>1.147791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>-2.300481</td>\n",
       "      <td>1.023173</td>\n",
       "      <td>0.449162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>-1.793062</td>\n",
       "      <td>2.209649</td>\n",
       "      <td>-0.045031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>-0.221409</td>\n",
       "      <td>1.688750</td>\n",
       "      <td>0.069356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>0.141471</td>\n",
       "      <td>-1.880294</td>\n",
       "      <td>1.984397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>-0.359952</td>\n",
       "      <td>1.141758</td>\n",
       "      <td>2.019301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.171312</td>\n",
       "      <td>0.079332</td>\n",
       "      <td>0.178522</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>750 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            d         e         f\n",
       "162  1.315442 -1.039258  0.194516\n",
       "168 -1.908995 -0.953551 -1.430472\n",
       "214  0.181162  1.022858 -2.289700\n",
       "895  2.825765 -1.205520  1.147791\n",
       "154 -2.300481  1.023173  0.449162\n",
       "..        ...       ...       ...\n",
       "32  -1.793062  2.209649 -0.045031\n",
       "829 -0.221409  1.688750  0.069356\n",
       "176  0.141471 -1.880294  1.984397\n",
       "124 -0.359952  1.141758  2.019301\n",
       "35   0.171312  0.079332  0.178522\n",
       "\n",
       "[750 rows x 3 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tpot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import sklearn\n",
    "\n",
    "subsets = { \"group_one\" :  ['a','b','c'],\n",
    "            \"group_two\" :  ['d','e','f'],\n",
    "            \"group_three\" :  ['g','h','i'],\n",
    "            }\n",
    "\n",
    "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n",
    "\n",
    "selector = fss_search_space.generate(rng=1).export_pipeline()\n",
    "selector.set_output(transform=\"pandas\")\n",
    "selector.fit(X_train)\n",
    "selector.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.315442</td>\n",
       "      <td>-1.039258</td>\n",
       "      <td>0.194516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>-1.908995</td>\n",
       "      <td>-0.953551</td>\n",
       "      <td>-1.430472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>0.181162</td>\n",
       "      <td>1.022858</td>\n",
       "      <td>-2.289700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2.825765</td>\n",
       "      <td>-1.205520</td>\n",
       "      <td>1.147791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>-2.300481</td>\n",
       "      <td>1.023173</td>\n",
       "      <td>0.449162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>-1.793062</td>\n",
       "      <td>2.209649</td>\n",
       "      <td>-0.045031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>-0.221409</td>\n",
       "      <td>1.688750</td>\n",
       "      <td>0.069356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>0.141471</td>\n",
       "      <td>-1.880294</td>\n",
       "      <td>1.984397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>-0.359952</td>\n",
       "      <td>1.141758</td>\n",
       "      <td>2.019301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.171312</td>\n",
       "      <td>0.079332</td>\n",
       "      <td>0.178522</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>750 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            d         e         f\n",
       "162  1.315442 -1.039258  0.194516\n",
       "168 -1.908995 -0.953551 -1.430472\n",
       "214  0.181162  1.022858 -2.289700\n",
       "895  2.825765 -1.205520  1.147791\n",
       "154 -2.300481  1.023173  0.449162\n",
       "..        ...       ...       ...\n",
       "32  -1.793062  2.209649 -0.045031\n",
       "829 -0.221409  1.688750  0.069356\n",
       "176  0.141471 -1.880294  1.984397\n",
       "124 -0.359952  1.141758  2.019301\n",
       "35   0.171312  0.079332  0.178522\n",
       "\n",
       "[750 rows x 3 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tpot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import sklearn\n",
    "\n",
    "subsets = [['a','b','c'],['d','e','f'],['g','h','i']]\n",
    "\n",
    "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n",
    "\n",
    "selector = fss_search_space.generate(rng=1).export_pipeline()\n",
    "selector.set_output(transform=\"pandas\")\n",
    "selector.fit(X_train)\n",
    "selector.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## csv file\n",
    "\n",
    "note: watch for spaces in the csv file!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>1.315442</td>\n",
       "      <td>-1.039258</td>\n",
       "      <td>0.194516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>-1.908995</td>\n",
       "      <td>-0.953551</td>\n",
       "      <td>-1.430472</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>0.181162</td>\n",
       "      <td>1.022858</td>\n",
       "      <td>-2.289700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>895</th>\n",
       "      <td>2.825765</td>\n",
       "      <td>-1.205520</td>\n",
       "      <td>1.147791</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>-2.300481</td>\n",
       "      <td>1.023173</td>\n",
       "      <td>0.449162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>-1.793062</td>\n",
       "      <td>2.209649</td>\n",
       "      <td>-0.045031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>829</th>\n",
       "      <td>-0.221409</td>\n",
       "      <td>1.688750</td>\n",
       "      <td>0.069356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176</th>\n",
       "      <td>0.141471</td>\n",
       "      <td>-1.880294</td>\n",
       "      <td>1.984397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>-0.359952</td>\n",
       "      <td>1.141758</td>\n",
       "      <td>2.019301</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>0.171312</td>\n",
       "      <td>0.079332</td>\n",
       "      <td>0.178522</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>750 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            d         e         f\n",
       "162  1.315442 -1.039258  0.194516\n",
       "168 -1.908995 -0.953551 -1.430472\n",
       "214  0.181162  1.022858 -2.289700\n",
       "895  2.825765 -1.205520  1.147791\n",
       "154 -2.300481  1.023173  0.449162\n",
       "..        ...       ...       ...\n",
       "32  -1.793062  2.209649 -0.045031\n",
       "829 -0.221409  1.688750  0.069356\n",
       "176  0.141471 -1.880294  1.984397\n",
       "124 -0.359952  1.141758  2.019301\n",
       "35   0.171312  0.079332  0.178522\n",
       "\n",
       "[750 rows x 3 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tpot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import sklearn\n",
    "\n",
    "subsets = 'simple_fss.csv'\n",
    "'''\n",
    "# simple_fss.csv\n",
    "one,a,b,c\n",
    "two,d,e,f\n",
    "three,g,h,i\n",
    "'''\n",
    "\n",
    "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n",
    "\n",
    "selector = fss_search_space.generate(rng=1).export_pipeline()\n",
    "selector.set_output(transform=\"pandas\")\n",
    "selector.fit(X_train)\n",
    "selector.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of the above is the same when using numpy data, but the column names are replaced int indexes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.31748616  2.20805859 -2.21719911 ...  0.5595234   0.80605806\n",
      "   0.41484993]\n",
      " [ 2.8673731   1.45905176 -1.11516833 ...  0.74646156  0.95635356\n",
      "   0.03575697]\n",
      " [-1.64867116  2.14478724  2.31196119 ...  0.22969172  0.72447325\n",
      "   0.81842014]\n",
      " ...\n",
      " [ 1.17772695  0.7188885  -0.52548496 ...  0.99266968  0.95436462\n",
      "   0.57430922]\n",
      " [ 0.14052568  0.15042817 -0.86281564 ...  0.25379746  0.1818071\n",
      "   0.55993116]\n",
      " [ 1.37273916 -0.14898886 -0.89938251 ...  0.767549    0.66184827\n",
      "   0.49174333]]\n"
     ]
    }
   ],
   "source": [
    "import tpot\n",
    "import sklearn.datasets\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "n_features = 6\n",
    "X, y = sklearn.datasets.make_classification(n_samples=1000, n_features=n_features, n_informative=6, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None)\n",
    "X = np.hstack([X, np.random.rand(X.shape[0],3)]) #add three uninformative features\n",
    "\n",
    "X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(X, y, train_size=0.75, test_size=0.25)\n",
    "\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.76235619, -1.97629642,  1.05447979],\n",
       "       [ 2.16944118, -1.55515714,  0.67925075],\n",
       "       [ 1.96557199,  0.13789923,  1.588271  ],\n",
       "       ...,\n",
       "       [ 0.78956322,  2.12535053,  0.63115798],\n",
       "       [-0.80184984, -0.40793866,  1.3880617 ],\n",
       "       [-1.38085267,  1.62568989, -1.42046795]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tpot\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "import sklearn\n",
    "\n",
    "subsets = { \"group_one\" :  [0,1,2],\n",
    "            \"group_two\" :  [3,4,5],\n",
    "            \"group_three\" :  [6,7,8],\n",
    "            }\n",
    "\n",
    "fss_search_space = tpot.search_spaces.nodes.FSSNode(subsets=subsets)\n",
    "selector = fss_search_space.generate(rng=1).export_pipeline()\n",
    "selector.fit(X_train)\n",
    "selector.transform(X_train)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tpotenv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.16"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
